You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2016/07/03 18:07:11 UTC

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 691 - Still Failing!

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/691/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseSerialGC

7 tests failed.
FAILED:  org.apache.solr.cloud.overseer.ZkStateReaderTest.testExternalCollectionWatchedNotWatched

Error Message:


Stack Trace:
java.lang.NullPointerException
	at __randomizedtesting.SeedInfo.seed([39A296B9D7326295:32196795866B943C]:0)
	at org.apache.solr.cloud.overseer.ZkStateReaderTest.testExternalCollectionWatchedNotWatched(ZkStateReaderTest.java:167)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.overseer.ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy

Error Message:
Could not find collection : c1

Stack Trace:
org.apache.solr.common.SolrException: Could not find collection : c1
	at __randomizedtesting.SeedInfo.seed([39A296B9D7326295:52ED36C4AE3DBFAF]:0)
	at org.apache.solr.common.cloud.ClusterState.getCollection(ClusterState.java:192)
	at org.apache.solr.cloud.overseer.ZkStateReaderTest.testStateFormatUpdate(ZkStateReaderTest.java:129)
	at org.apache.solr.cloud.overseer.ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy(ZkStateReaderTest.java:48)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.overseer.ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy

Error Message:
expected:<2> but was:<1>

Stack Trace:
java.lang.AssertionError: expected:<2> but was:<1>
	at __randomizedtesting.SeedInfo.seed([39A296B9D7326295:9659B8987319DBA9]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.failNotEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:128)
	at org.junit.Assert.assertEquals(Assert.java:472)
	at org.junit.Assert.assertEquals(Assert.java:456)
	at org.apache.solr.cloud.overseer.ZkStateReaderTest.testStateFormatUpdate(ZkStateReaderTest.java:130)
	at org.apache.solr.cloud.overseer.ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy(ZkStateReaderTest.java:58)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.overseer.ZkStateWriterTest.testExternalModificationToStateFormat2

Error Message:


Stack Trace:
java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([39A296B9D7326295:487F826DB6999E15]:0)
	at org.junit.Assert.fail(Assert.java:92)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertTrue(Assert.java:54)
	at org.apache.solr.cloud.overseer.ZkStateWriterTest.testExternalModificationToStateFormat2(ZkStateWriterTest.java:322)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.handler.TestReqParamsAPI.test

Error Message:
Could not get expected value  'CY val' for path 'params/c' full output: {   "responseHeader":{     "status":0,     "QTime":0},   "params":{     "a":"A val",     "b":"B val",     "wt":"json",     "useParams":""},   "context":{     "webapp":"/iwea/b",     "path":"/dump1",     "httpMethod":"GET"}},  from server:  http://127.0.0.1:52778/iwea/b/collection1

Stack Trace:
java.lang.AssertionError: Could not get expected value  'CY val' for path 'params/c' full output: {
  "responseHeader":{
    "status":0,
    "QTime":0},
  "params":{
    "a":"A val",
    "b":"B val",
    "wt":"json",
    "useParams":""},
  "context":{
    "webapp":"/iwea/b",
    "path":"/dump1",
    "httpMethod":"GET"}},  from server:  http://127.0.0.1:52778/iwea/b/collection1
	at __randomizedtesting.SeedInfo.seed([39A296B9D7326295:B1F6A96379CE0F6D]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.core.TestSolrConfigHandler.testForResponseElement(TestSolrConfigHandler.java:481)
	at org.apache.solr.handler.TestReqParamsAPI.testReqParams(TestReqParamsAPI.java:171)
	at org.apache.solr.handler.TestReqParamsAPI.test(TestReqParamsAPI.java:61)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.update.AutoCommitTest.testCommitWithin

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([39A296B9D7326295:8370F9C1541C8C80]:0)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:780)
	at org.apache.solr.update.AutoCommitTest.testCommitWithin(AutoCommitTest.java:325)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
	xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
</response>

	request was:q=id:529&qt=standard&start=0&rows=20&version=2.2
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:773)
	... 40 more


FAILED:  org.apache.solr.cloud.overseer.ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh

Error Message:
Could not find collection : c1

Stack Trace:
org.apache.solr.common.SolrException: Could not find collection : c1
	at __randomizedtesting.SeedInfo.seed([39A296B9D7326295:2618E74E0752A450]:0)
	at org.apache.solr.common.cloud.ClusterState.getCollection(ClusterState.java:192)
	at org.apache.solr.cloud.overseer.ZkStateReaderTest.testStateFormatUpdate(ZkStateReaderTest.java:129)
	at org.apache.solr.cloud.overseer.ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh(ZkStateReaderTest.java:43)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10740 lines...]
   [junit4] Suite: org.apache.solr.cloud.overseer.ZkStateReaderTest
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.overseer.ZkStateReaderTest_39A296B9D7326295-001/init-core-data-001
   [junit4]   2> 356805 INFO  (SUITE-ZkStateReaderTest-seed#[39A296B9D7326295]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 356806 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[39A296B9D7326295]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testExternalCollectionWatchedNotWatched
   [junit4]   2> 356807 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[39A296B9D7326295]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 356807 INFO  (Thread-4111) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 356807 INFO  (Thread-4111) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 356907 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[39A296B9D7326295]) [    ] o.a.s.c.ZkTestServer start zk server on port:43094
   [junit4]   2> 356907 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 356908 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 356910 INFO  (zkCallback-471-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2c9af299 name:ZooKeeperConnection Watcher:127.0.0.1:43094 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 356910 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 356910 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 356912 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 356912 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 356914 INFO  (zkCallback-472-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@454641b7 name:ZooKeeperConnection Watcher:127.0.0.1:43094 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 356914 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 356914 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 356914 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 356922 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 356922 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 356924 INFO  (zkCallback-473-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@14a687d3 name:ZooKeeperConnection Watcher:127.0.0.1:43094/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 356924 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 356924 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 356925 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 356927 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections
   [junit4]   2> 356930 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 356931 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 356934 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 356935 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 356937 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 356939 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 356940 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/c1
   [junit4]   2> 356945 INFO  (zkCallback-473-thread-1) [    ] o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections], has occurred - updating...
   [junit4]   2> 356947 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[39A296B9D7326295]) [    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/c1/state.json
   [junit4]   2> 356962 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[39A296B9D7326295]) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:43094 43094
   [junit4]   2> 357086 INFO  (Thread-4111) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:43094 43094
   [junit4]   2> 357094 INFO  (TEST-ZkStateReaderTest.testExternalCollectionWatchedNotWatched-seed#[39A296B9D7326295]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testExternalCollectionWatchedNotWatched
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=ZkStateReaderTest -Dtests.method=testExternalCollectionWatchedNotWatched -Dtests.seed=39A296B9D7326295 -Dtests.slow=true -Dtests.locale=ga -Dtests.timezone=America/St_Barthelemy -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.29s J0 | ZkStateReaderTest.testExternalCollectionWatchedNotWatched <<<
   [junit4]    > Throwable #1: java.lang.NullPointerException
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([39A296B9D7326295:32196795866B943C]:0)
   [junit4]    > 	at org.apache.solr.cloud.overseer.ZkStateReaderTest.testExternalCollectionWatchedNotWatched(ZkStateReaderTest.java:167)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 357099 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[39A296B9D7326295]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testStateFormatUpdateWithTimeDelay
   [junit4]   2> 357099 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[39A296B9D7326295]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 357099 INFO  (Thread-4112) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 357099 INFO  (Thread-4112) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 357200 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[39A296B9D7326295]) [    ] o.a.s.c.ZkTestServer start zk server on port:45820
   [junit4]   2> 357200 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 357200 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 357203 INFO  (zkCallback-475-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@29f1dc6c name:ZooKeeperConnection Watcher:127.0.0.1:45820 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 357203 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 357203 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 357205 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 357205 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 357206 INFO  (zkCallback-476-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@35be492c name:ZooKeeperConnection Watcher:127.0.0.1:45820 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 357207 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 357207 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 357207 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 357210 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 357210 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 357212 INFO  (zkCallback-477-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7d3148e name:ZooKeeperConnection Watcher:127.0.0.1:45820/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 357212 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 357212 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 357213 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 357214 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections
   [junit4]   2> 357217 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 357219 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 357220 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 357222 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 357222 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 357223 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 357226 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ZkStateReader Deleting data for [c1]
   [junit4]   2> 357226 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/c1
   [junit4]   2> 357228 INFO  (zkCallback-477-thread-1) [    ] o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections], has occurred - updating...
   [junit4]   2> 357233 INFO  (zkCallback-477-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 357240 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[39A296B9D7326295]) [    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/c1/state.json
   [junit4]   2> 357241 INFO  (zkCallback-477-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeCreated path:/collections/c1/state.json] for collection [c1] has occurred - updating... (live nodes size: [0])
   [junit4]   2> 357241 INFO  (zkCallback-477-thread-1) [    ] o.a.s.c.c.ZkStateReader Add data for [c1] ver [0]
   [junit4]   2> 357242 INFO  (zkCallback-477-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 357245 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[39A296B9D7326295]) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:45820 45820
   [junit4]   2> 357378 INFO  (Thread-4112) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:45820 45820
   [junit4]   2> 357379 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelay-seed#[39A296B9D7326295]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testStateFormatUpdateWithTimeDelay
   [junit4]   2> 357382 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[39A296B9D7326295]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testWatchedCollectionCreation
   [junit4]   2> 357382 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[39A296B9D7326295]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 357382 INFO  (Thread-4113) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 357382 INFO  (Thread-4113) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 357482 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[39A296B9D7326295]) [    ] o.a.s.c.ZkTestServer start zk server on port:48049
   [junit4]   2> 357482 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 357484 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 357486 INFO  (zkCallback-479-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2066735c name:ZooKeeperConnection Watcher:127.0.0.1:48049 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 357486 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 357487 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 357488 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 357489 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 357490 INFO  (zkCallback-480-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@662cf9d2 name:ZooKeeperConnection Watcher:127.0.0.1:48049 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 357490 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 357490 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 357490 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 357495 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 357496 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 357497 INFO  (zkCallback-481-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4f8d0e9d name:ZooKeeperConnection Watcher:127.0.0.1:48049/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 357497 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 357497 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 357499 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 357500 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections
   [junit4]   2> 357507 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 357509 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 357511 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 357512 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 357514 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 357515 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 357518 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ZkStateReader Deleting data for [c1]
   [junit4]   2> 357518 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/c1
   [junit4]   2> 357521 INFO  (zkCallback-481-thread-1) [    ] o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections], has occurred - updating...
   [junit4]   2> 357525 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[39A296B9D7326295]) [    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/c1/state.json
   [junit4]   2> 357526 INFO  (zkCallback-481-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeCreated path:/collections/c1/state.json] for collection [c1] has occurred - updating... (live nodes size: [0])
   [junit4]   2> 357527 INFO  (zkCallback-481-thread-2) [    ] o.a.s.c.c.ZkStateReader Add data for [c1] ver [0]
   [junit4]   2> 357531 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[39A296B9D7326295]) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:48049 48049
   [junit4]   2> 357555 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxnFactory Ignoring exception
   [junit4]   2> java.nio.channels.ClosedChannelException
   [junit4]   2> 	at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:235)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:188)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 357556 INFO  (Thread-4113) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:48049 48049
   [junit4]   2> 357558 INFO  (TEST-ZkStateReaderTest.testWatchedCollectionCreation-seed#[39A296B9D7326295]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testWatchedCollectionCreation
   [junit4]   2> 357563 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[39A296B9D7326295]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testStateFormatUpdateWithExplicitRefreshLazy
   [junit4]   2> 357564 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 357565 INFO  (Thread-4114) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 357566 INFO  (Thread-4114) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 357664 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.ZkTestServer start zk server on port:33518
   [junit4]   2> 357664 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 357665 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 357670 INFO  (zkCallback-483-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@9726119 name:ZooKeeperConnection Watcher:127.0.0.1:33518 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 357671 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 357671 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 357675 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 357676 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 357684 INFO  (zkCallback-484-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@594e81f8 name:ZooKeeperConnection Watcher:127.0.0.1:33518 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 357684 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 357686 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 357686 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 357693 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 357693 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 357697 INFO  (zkCallback-485-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@47e3a2ec name:ZooKeeperConnection Watcher:127.0.0.1:33518/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 357697 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 357697 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 357698 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 357701 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections
   [junit4]   2> 357709 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 357712 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 357714 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 357715 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 357715 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 357716 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 357717 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/c1
   [junit4]   2> 357720 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/c1/state.json
   [junit4]   2> 357723 ERROR (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[39A296B9D7326295]-EventThread) [    ] o.a.z.ClientCnxn Error while calling watcher 
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$8/1847090701@23ebe6d6 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@27d46673[Shutting down, pool size = 1, active threads = 0, queued tasks = 0, completed tasks = 1]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2047)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:823)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1369)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:214)
   [junit4]   2> 	at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.process(SolrZkClient.java:267)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
   [junit4]   2> 357723 ERROR (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[39A296B9D7326295]-EventThread) [    ] o.a.z.ClientCnxn Error while calling watcher 
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$8/1847090701@2f5fb1a1 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@27d46673[Shutting down, pool size = 1, active threads = 0, queued tasks = 0, completed tasks = 1]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2047)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:823)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1369)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:214)
   [junit4]   2> 	at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.process(SolrZkClient.java:267)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
   [junit4]   2> 357743 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:33518 33518
   [junit4]   2> 357755 INFO  (Thread-4114) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:33518 33518
   [junit4]   2> 357759 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy-seed#[39A296B9D7326295]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testStateFormatUpdateWithExplicitRefreshLazy
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=ZkStateReaderTest -Dtests.method=testStateFormatUpdateWithExplicitRefreshLazy -Dtests.seed=39A296B9D7326295 -Dtests.slow=true -Dtests.locale=ga -Dtests.timezone=America/St_Barthelemy -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.20s J0 | ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy <<<
   [junit4]    > Throwable #1: org.apache.solr.common.SolrException: Could not find collection : c1
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([39A296B9D7326295:52ED36C4AE3DBFAF]:0)
   [junit4]    > 	at org.apache.solr.common.cloud.ClusterState.getCollection(ClusterState.java:192)
   [junit4]    > 	at org.apache.solr.cloud.overseer.ZkStateReaderTest.testStateFormatUpdate(ZkStateReaderTest.java:129)
   [junit4]    > 	at org.apache.solr.cloud.overseer.ZkStateReaderTest.testStateFormatUpdateWithExplicitRefreshLazy(ZkStateReaderTest.java:48)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 357765 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[39A296B9D7326295]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testStateFormatUpdateWithExplicitRefresh
   [junit4]   2> 357766 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[39A296B9D7326295]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 357766 INFO  (Thread-4115) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 357766 INFO  (Thread-4115) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 357866 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[39A296B9D7326295]) [    ] o.a.s.c.ZkTestServer start zk server on port:40691
   [junit4]   2> 357866 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 357867 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 357870 INFO  (zkCallback-487-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@24228954 name:ZooKeeperConnection Watcher:127.0.0.1:40691 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 357870 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 357871 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 357878 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 357879 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 357884 INFO  (zkCallback-488-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5e68829d name:ZooKeeperConnection Watcher:127.0.0.1:40691 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 357884 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 357884 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 357884 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 357903 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 357904 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 357907 INFO  (zkCallback-489-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@17e0724c name:ZooKeeperConnection Watcher:127.0.0.1:40691/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 357907 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 357907 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 357908 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 357910 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections
   [junit4]   2> 357912 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 357922 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 357927 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 357929 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 357930 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 357932 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 357934 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ZkStateReader Deleting data for [c1]
   [junit4]   2> 357934 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/c1
   [junit4]   2> 357938 INFO  (zkCallback-489-thread-1) [    ] o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections], has occurred - updating...
   [junit4]   2> 357943 INFO  (zkCallback-489-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 357947 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[39A296B9D7326295]) [    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/c1/state.json
   [junit4]   2> 357953 INFO  (zkCallback-489-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 357956 INFO  (zkCallback-489-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeCreated path:/collections/c1/state.json] for collection [c1] has occurred - updating... (live nodes size: [0])
   [junit4]   2> 357956 WARN  (zkCallback-489-thread-1) [    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /collections/c1/state.json]
   [junit4]   2> 357965 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[39A296B9D7326295]) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:40691 40691
   [junit4]   2> 358098 INFO  (Thread-4115) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:40691 40691
   [junit4]   2> 358099 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh-seed#[39A296B9D7326295]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testStateFormatUpdateWithExplicitRefresh
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=ZkStateReaderTest -Dtests.method=testStateFormatUpdateWithExplicitRefresh -Dtests.seed=39A296B9D7326295 -Dtests.slow=true -Dtests.locale=ga -Dtests.timezone=America/St_Barthelemy -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.34s J0 | ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh <<<
   [junit4]    > Throwable #1: org.apache.solr.common.SolrException: Could not find collection : c1
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([39A296B9D7326295:2618E74E0752A450]:0)
   [junit4]    > 	at org.apache.solr.common.cloud.ClusterState.getCollection(ClusterState.java:192)
   [junit4]    > 	at org.apache.solr.cloud.overseer.ZkStateReaderTest.testStateFormatUpdate(ZkStateReaderTest.java:129)
   [junit4]    > 	at org.apache.solr.cloud.overseer.ZkStateReaderTest.testStateFormatUpdateWithExplicitRefresh(ZkStateReaderTest.java:43)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 358102 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[39A296B9D7326295]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testStateFormatUpdateWithTimeDelayLazy
   [junit4]   2> 358102 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 358103 INFO  (Thread-4116) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 358103 INFO  (Thread-4116) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 358203 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.ZkTestServer start zk server on port:34925
   [junit4]   2> 358203 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 358204 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 358218 INFO  (zkCallback-491-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@d54f597 name:ZooKeeperConnection Watcher:127.0.0.1:34925 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 358218 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 358218 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 358229 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 358230 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 358251 INFO  (zkCallback-492-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@81c4161 name:ZooKeeperConnection Watcher:127.0.0.1:34925 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 358255 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 358255 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 358255 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 358264 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 358266 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 358279 INFO  (zkCallback-493-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2d5c5501 name:ZooKeeperConnection Watcher:127.0.0.1:34925/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 358279 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 358280 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 358280 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 358285 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections
   [junit4]   2> 358293 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 358295 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 358297 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 358298 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 358299 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 358299 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 358301 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/c1
   [junit4]   2> 358305 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ZkStateReader Deleting data for [c1]
   [junit4]   2> 358305 INFO  (zkCallback-493-thread-1) [    ] o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections], has occurred - updating...
   [junit4]   2> 358319 INFO  (zkCallback-493-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 358334 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/c1/state.json
   [junit4]   2> 358336 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ZkStateReader Add data for [c1] ver [0]
   [junit4]   2> 358338 ERROR (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[39A296B9D7326295]-EventThread) [    ] o.a.z.ClientCnxn Error while calling watcher 
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$8/1847090701@47a3e80b rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@2a17e9b[Shutting down, pool size = 2, active threads = 0, queued tasks = 0, completed tasks = 3]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2047)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:823)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1369)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:214)
   [junit4]   2> 	at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.process(SolrZkClient.java:267)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
   [junit4]   2> 358339 ERROR (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[39A296B9D7326295]-EventThread) [    ] o.a.z.ClientCnxn Error while calling watcher 
   [junit4]   2> java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$8/1847090701@33693a55 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@2a17e9b[Shutting down, pool size = 2, active threads = 0, queued tasks = 0, completed tasks = 3]
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2047)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:823)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1369)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:214)
   [junit4]   2> 	at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.process(SolrZkClient.java:267)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
   [junit4]   2> 358356 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[39A296B9D7326295]) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:34925 34925
   [junit4]   2> 358381 INFO  (Thread-4116) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:34925 34925
   [junit4]   2> 358383 INFO  (TEST-ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy-seed#[39A296B9D7326295]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testStateFormatUpdateWithTimeDelayLazy
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=ZkStateReaderTest -Dtests.method=testStateFormatUpdateWithTimeDelayLazy -Dtests.seed=39A296B9D7326295 -Dtests.slow=true -Dtests.locale=ga -Dtests.timezone=America/St_Barthelemy -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] FAILURE 0.28s J0 | ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: expected:<2> but was:<1>
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([39A296B9D7326295:9659B8987319DBA9]:0)
   [junit4]    > 	at org.apache.solr.cloud.overseer.ZkStateReaderTest.testStateFormatUpdate(ZkStateReaderTest.java:130)
   [junit4]    > 	at org.apache.solr.cloud.overseer.ZkStateReaderTest.testStateFormatUpdateWithTimeDelayLazy(ZkStateReaderTest.java:58)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 358384 INFO  (SUITE-ZkStateReaderTest-seed#[39A296B9D7326295]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.overseer.ZkStateReaderTest_39A296B9D7326295-001
   [junit4]   2> Jul 03, 2016 5:21:39 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 4 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene62): {}, docValues:{}, maxPointsInLeafNode=1188, maxMBSortInHeap=7.702391912581557, sim=RandomSimilarity(queryNorm=false,coord=crazy): {}, locale=ga, timezone=America/St_Barthelemy
   [junit4]   2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_92 (64-bit)/cpus=3,threads=1,free=78833648,total=247332864
   [junit4]   2> NOTE: All tests run in this JVM: [TestCSVLoader, TestExceedMaxTermLength, SolrXmlInZkTest, PKIAuthenticationIntegrationTest, TestHashQParserPlugin, ZkCLITest, SystemInfoHandlerTest, PolyFieldTest, FastVectorHighlighterTest, TestSortByMinMaxFunction, TimeZoneUtilsTest, TriLevelCompositeIdRoutingTest, HighlighterMaxOffsetTest, QueryElevationComponentTest, TestReplicaProperties, TestExtendedDismaxParser, StandardRequestHandlerTest, PrimitiveFieldTypeTest, CollectionStateFormat2Test, TestSha256AuthenticationProvider, HardAutoCommitTest, TestMergePolicyConfig, TestSubQueryTransformerCrossCore, MoreLikeThisHandlerTest, TestElisionMultitermQuery, TestDFRSimilarityFactory, DistanceUnitsTest, HdfsRestartWhileUpdatingTest, SolrRequestParserTest, TestSurroundQueryParser, TestDocumentBuilder, TestNumericTerms32, TestGroupingSearch, DocValuesMultiTest, TestCursorMarkWithoutUniqueKey, TestConfigSetImmutable, AnalysisAfterCoreReloadTest, ConjunctionSolrSpellCheckerTest, TestSchemaResource, DistributedFacetPivotSmallTest, SchemaVersionSpecificBehaviorTest, BlockCacheTest, ReturnFieldsTest, SoftAutoCommitTest, TestStressReorder, TestRecoveryHdfs, TestSolrQueryParserDefaultOperatorResource, MigrateRouteKeyTest, TestCustomSort, DistribCursorPagingTest, ZkStateReaderTest]
   [junit4] Completed [88/618 (1!)] on J0 in 2.10s, 6 tests, 1 failure, 3 errors <<< FAILURES!

[...truncated 229 lines...]
   [junit4] Suite: org.apache.solr.handler.TestReqParamsAPI
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.handler.TestReqParamsAPI_39A296B9D7326295-001/init-core-data-001
   [junit4]   2> 710475 INFO  (SUITE-TestReqParamsAPI-seed#[39A296B9D7326295]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 710475 INFO  (SUITE-TestReqParamsAPI-seed#[39A296B9D7326295]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /iwea/b
   [junit4]   2> 710480 INFO  (TEST-TestReqParamsAPI.test-seed#[39A296B9D7326295]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 710480 INFO  (Thread-5359) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 710480 INFO  (Thread-5359) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 710580 INFO  (TEST-TestReqParamsAPI.test-seed#[39A296B9D7326295]) [    ] o.a.s.c.ZkTestServer start zk server on port:40167
   [junit4]   2> 710580 INFO  (TEST-TestReqParamsAPI.test-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 710581 INFO  (TEST-TestReqParamsAPI.test-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 710583 INFO  (zkCallback-1059-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@57f73e24 name:ZooKeeperConnection Watcher:127.0.0.1:40167 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 710584 INFO  (TEST-TestReqParamsAPI.test-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 710584 INFO  (TEST-TestReqParamsAPI.test-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 710584 INFO  (TEST-TestReqParamsAPI.test-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 710587 INFO  (TEST-TestReqParamsAPI.test-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 710587 INFO  (TEST-TestReqParamsAPI.test-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 710588 INFO  (zkCallback-1060-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@48bc3e47 name:ZooKeeperConnection Watcher:127.0.0.1:40167/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 710588 INFO  (TEST-TestReqParamsAPI.test-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 710589 INFO  (TEST-TestReqParamsAPI.test-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 710589 INFO  (TEST-TestReqParamsAPI.test-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 710592 INFO  (TEST-TestReqParamsAPI.test-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 710594 INFO  (TEST-TestReqParamsAPI.test-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 710596 INFO  (TEST-TestReqParamsAPI.test-seed#[39A296B9D7326295]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 710598 INFO  (TEST-TestReqParamsAPI.test-seed#[39A296B9D7326295]) [    ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solr

[...truncated too long message...]

action_logs_total_number=0}
   [junit4]   2> 1840346 INFO  (coreCloseExecutor-6738-thread-1-processing-x:collection1) [    x:collection1] o.a.s.u.SolrCoreState Closing SolrCoreState
   [junit4]   2> 1840346 INFO  (coreCloseExecutor-6738-thread-1-processing-x:collection1) [    x:collection1] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 1840346 INFO  (coreCloseExecutor-6738-thread-1-processing-x:collection1) [    x:collection1] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
   [junit4]   2> 1840346 INFO  (coreCloseExecutor-6738-thread-1-processing-x:collection1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter close.
   [junit4]   2> 1840351 INFO  (coreCloseExecutor-6738-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@bf7b934 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7d252fb5),segFN=segments_3,generation=3}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@bf7b934 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7d252fb5),segFN=segments_4,generation=4}
   [junit4]   2> 1840351 INFO  (coreCloseExecutor-6738-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 4
   [junit4]   2> 1840352 INFO  (coreCloseExecutor-6738-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 1840383 INFO  (coreCloseExecutor-6738-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 1840384 INFO  (coreCloseExecutor-6738-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.update.AutoCommitTest_39A296B9D7326295-001/init-core-data-001 [CachedDir<<refCount=0;path=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.update.AutoCommitTest_39A296B9D7326295-001/init-core-data-001;done=false>>]
   [junit4]   2> 1840384 INFO  (coreCloseExecutor-6738-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory, CoreContainer#isShutdown=true
   [junit4]   2> 1840384 INFO  (coreCloseExecutor-6738-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.update.AutoCommitTest_39A296B9D7326295-001/init-core-data-001
   [junit4]   2> 1840384 INFO  (coreCloseExecutor-6738-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.update.AutoCommitTest_39A296B9D7326295-001/init-core-data-001/index [CachedDir<<refCount=0;path=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.update.AutoCommitTest_39A296B9D7326295-001/init-core-data-001/index;done=false>>]
   [junit4]   2> 1840384 INFO  (coreCloseExecutor-6738-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory, CoreContainer#isShutdown=true
   [junit4]   2> 1840384 INFO  (coreCloseExecutor-6738-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.update.AutoCommitTest_39A296B9D7326295-001/init-core-data-001/index
   [junit4]   2> NOTE: leaving temporary files on disk at: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.update.AutoCommitTest_39A296B9D7326295-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene62): {range_facet_l_dv=PostingsFormat(name=Asserting), _version_=FST50, field_t=FSTOrd50, multiDefault=PostingsFormat(name=Memory doPackFST= true), subject=PostingsFormat(name=Memory doPackFST= true), intDefault=FST50, id=PostingsFormat(name=Asserting), range_facet_i_dv=FST50, text=FSTOrd50, range_facet_l=FST50, timestamp=FST50}, docValues:{range_facet_l_dv=DocValuesFormat(name=Lucene54), range_facet_i_dv=DocValuesFormat(name=Direct), timestamp=DocValuesFormat(name=Direct)}, maxPointsInLeafNode=467, maxMBSortInHeap=5.206570004852505, sim=ClassicSimilarity, locale=ar-IQ, timezone=Asia/Krasnoyarsk
   [junit4]   2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_92 (64-bit)/cpus=3,threads=1,free=174597624,total=518979584
   [junit4]   2> NOTE: All tests run in this JVM: [TestCSVLoader, TestExceedMaxTermLength, SolrXmlInZkTest, PKIAuthenticationIntegrationTest, TestHashQParserPlugin, ZkCLITest, SystemInfoHandlerTest, PolyFieldTest, FastVectorHighlighterTest, TestSortByMinMaxFunction, TimeZoneUtilsTest, TriLevelCompositeIdRoutingTest, HighlighterMaxOffsetTest, QueryElevationComponentTest, TestReplicaProperties, TestExtendedDismaxParser, StandardRequestHandlerTest, PrimitiveFieldTypeTest, CollectionStateFormat2Test, TestSha256AuthenticationProvider, HardAutoCommitTest, TestMergePolicyConfig, TestSubQueryTransformerCrossCore, MoreLikeThisHandlerTest, TestElisionMultitermQuery, TestDFRSimilarityFactory, DistanceUnitsTest, HdfsRestartWhileUpdatingTest, SolrRequestParserTest, TestSurroundQueryParser, TestDocumentBuilder, TestNumericTerms32, TestGroupingSearch, DocValuesMultiTest, TestCursorMarkWithoutUniqueKey, TestConfigSetImmutable, AnalysisAfterCoreReloadTest, ConjunctionSolrSpellCheckerTest, TestSchemaResource, DistributedFacetPivotSmallTest, SchemaVersionSpecificBehaviorTest, BlockCacheTest, ReturnFieldsTest, SoftAutoCommitTest, TestStressReorder, TestRecoveryHdfs, TestSolrQueryParserDefaultOperatorResource, MigrateRouteKeyTest, TestCustomSort, DistribCursorPagingTest, ZkStateReaderTest, ClusterStateTest, TestCollapseQParserPlugin, SharedFSAutoReplicaFailoverUtilsTest, TestLockTree, HdfsUnloadDistributedZkTest, TestJsonRequest, JsonLoaderTest, TestMiniSolrCloudClusterSSL, TestSolrXml, TestManagedSchemaAPI, PingRequestHandlerTest, TestStressVersions, BadComponentTest, DistributedSpellCheckComponentTest, TestFieldTypeResource, ConnectionManagerTest, IndexSchemaTest, TestQueryWrapperFilter, ExplicitHLLTest, TestBlobHandler, TestTolerantSearch, ShardRoutingTest, TestCloudBackupRestore, DOMUtilTest, CustomCollectionTest, HLLSerializationTest, TestFilteredDocIdSet, SSLMigrationTest, TestRestManager, TermsComponentTest, AlternateDirectoryTest, TestRTimerTree, TestFieldCacheWithThreads, HdfsRecoverLeaseTest, StatelessScriptUpdateProcessorFactoryTest, TestReqParamsAPI, ShardSplitTest, ChaosMonkeyNothingIsSafeTest, TestSmileRequest, TestCryptoKeys, TestScoreJoinQPNoScore, TestSearchPerf, TestSweetSpotSimilarityFactory, DistributedExpandComponentTest, TestManagedResource, SyncSliceTest, TestLRUCache, TestIBSimilarityFactory, IgnoreCommitOptimizeUpdateProcessorFactoryTest, TestDocSet, TestFunctionQuery, TestPostingsSolrHighlighter, TestDownShardTolerantSearch, SparseHLLTest, TestConfig, XsltUpdateRequestHandlerTest, TestSolrCloudWithKerberosAlt, TestDFISimilarityFactory, IndexBasedSpellCheckerTest, TestFaceting, TestUninvertingReader, DistributedSuggestComponentTest, AnalysisErrorHandlingTest, TestLRUStatsCache, PreAnalyzedUpdateProcessorTest, TestFastLRUCache, PathHierarchyTokenizerFactoryTest, SpatialHeatmapFacetsTest, DirectoryFactoryTest, BlockJoinFacetSimpleTest, TestPKIAuthenticationPlugin, TestXmlQParser, TestSort, TestPartialUpdateDeduplication, DocumentAnalysisRequestHandlerTest, SampleTest, OutOfBoxZkACLAndCredentialsProvidersTest, TestCodecSupport, HdfsLockFactoryTest, AnalyticsMergeStrategyTest, HttpPartitionTest, TestWordDelimiterFilterFactory, PrimUtilsTest, TestBulkSchemaConcurrent, RequiredFieldsTest, TestLeaderElectionZkExpiry, TestAnalyzedSuggestions, TestExpandComponent, ActionThrottleTest, TestBackupRepositoryFactory, XmlUpdateRequestHandlerTest, TestLazyCores, CollectionsAPIAsyncDistributedZkTest, CloudExitableDirectoryReaderTest, WrapperMergePolicyFactoryTest, InfoHandlerTest, StressHdfsTest, BasicDistributedZkTest, ChaosMonkeySafeLeaderTest, UnloadDistributedZkTest, BasicZkTest, RecoveryZkTest, TestReplicationHandler, AutoCommitTest]
   [junit4] Completed [322/618 (4!)] on J0 in 5.28s, 3 tests, 1 error <<< FAILURES!

[...truncated 992 lines...]
BUILD FAILED
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/build.xml:740: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/build.xml:684: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/build.xml:59: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build.xml:233: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/common-build.xml:530: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/lucene/common-build.xml:1427: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/lucene/common-build.xml:984: There were test failures: 618 suites (10 ignored), 2590 tests, 4 errors, 3 failures, 88 ignored (73 assumptions) [seed: 39A296B9D7326295]

Total time: 71 minutes 52 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 695 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/695/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseG1GC

1 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.client.solrj.embedded.LargeVolumeBinaryJettyTest

Error Message:
ObjectTracker found 1 object(s) that were not released!!! [TransactionLog]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 1 object(s) that were not released!!! [TransactionLog]
	at __randomizedtesting.SeedInfo.seed([2648413B1B80B622]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNull(Assert.java:551)
	at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:256)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 12995 lines...]
   [junit4] Suite: org.apache.solr.client.solrj.embedded.LargeVolumeBinaryJettyTest
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/init-core-data-001
   [junit4]   2> 83247 INFO  (SUITE-LargeVolumeBinaryJettyTest-seed#[2648413B1B80B622]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 83645 INFO  (SUITE-LargeVolumeBinaryJettyTest-seed#[2648413B1B80B622]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore
   [junit4]   2> 83645 INFO  (SUITE-LargeVolumeBinaryJettyTest-seed#[2648413B1B80B622]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore end
   [junit4]   2> 83645 INFO  (SUITE-LargeVolumeBinaryJettyTest-seed#[2648413B1B80B622]-worker) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-002/cores/core
   [junit4]   2> 83646 INFO  (SUITE-LargeVolumeBinaryJettyTest-seed#[2648413B1B80B622]-worker) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 83647 INFO  (SUITE-LargeVolumeBinaryJettyTest-seed#[2648413B1B80B622]-worker) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4874c9c4{/solr,null,AVAILABLE}
   [junit4]   2> 83651 INFO  (SUITE-LargeVolumeBinaryJettyTest-seed#[2648413B1B80B622]-worker) [    ] o.e.j.s.ServerConnector Started ServerConnector@4af6504d{HTTP/1.1,[http/1.1]}{127.0.0.1:38580}
   [junit4]   2> 83651 INFO  (SUITE-LargeVolumeBinaryJettyTest-seed#[2648413B1B80B622]-worker) [    ] o.e.j.s.Server Started @88705ms
   [junit4]   2> 83651 INFO  (SUITE-LargeVolumeBinaryJettyTest-seed#[2648413B1B80B622]-worker) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=38580, configSetBaseDir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001, coreRootDirectory=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-002/cores}
   [junit4]   2> 83651 INFO  (SUITE-LargeVolumeBinaryJettyTest-seed#[2648413B1B80B622]-worker) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6d06d69c
   [junit4]   2> 83651 INFO  (SUITE-LargeVolumeBinaryJettyTest-seed#[2648413B1B80B622]-worker) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001'
   [junit4]   2> 83652 INFO  (SUITE-LargeVolumeBinaryJettyTest-seed#[2648413B1B80B622]-worker) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 83652 INFO  (SUITE-LargeVolumeBinaryJettyTest-seed#[2648413B1B80B622]-worker) [    ] o.a.s.c.SolrResourceLoader using system property solr.solr.home: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001
   [junit4]   2> 83652 INFO  (SUITE-LargeVolumeBinaryJettyTest-seed#[2648413B1B80B622]-worker) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001/solr.xml
   [junit4]   2> 83657 INFO  (SUITE-LargeVolumeBinaryJettyTest-seed#[2648413B1B80B622]-worker) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001
   [junit4]   2> 83657 INFO  (SUITE-LargeVolumeBinaryJettyTest-seed#[2648413B1B80B622]-worker) [    ] o.a.s.c.CoreContainer New CoreContainer 2000744783
   [junit4]   2> 83657 INFO  (SUITE-LargeVolumeBinaryJettyTest-seed#[2648413B1B80B622]-worker) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001]
   [junit4]   2> 83658 WARN  (SUITE-LargeVolumeBinaryJettyTest-seed#[2648413B1B80B622]-worker) [    ] o.a.s.c.CoreContainer Couldn't add files from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001/lib to classpath: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001/lib
   [junit4]   2> 83658 INFO  (SUITE-LargeVolumeBinaryJettyTest-seed#[2648413B1B80B622]-worker) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 600000,connTimeout : 60000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,
   [junit4]   2> 83660 INFO  (SUITE-LargeVolumeBinaryJettyTest-seed#[2648413B1B80B622]-worker) [    ] o.a.s.u.UpdateShardHandler Created UpdateShardHandler HTTP client with params: socketTimeout=600000&connTimeout=60000&maxConnections=100000&maxConnectionsPerHost=100000
   [junit4]   2> 83660 INFO  (SUITE-LargeVolumeBinaryJettyTest-seed#[2648413B1B80B622]-worker) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 83661 INFO  (SUITE-LargeVolumeBinaryJettyTest-seed#[2648413B1B80B622]-worker) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 83661 INFO  (SUITE-LargeVolumeBinaryJettyTest-seed#[2648413B1B80B622]-worker) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 83661 INFO  (SUITE-LargeVolumeBinaryJettyTest-seed#[2648413B1B80B622]-worker) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 83662 INFO  (SUITE-LargeVolumeBinaryJettyTest-seed#[2648413B1B80B622]-worker) [    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001
   [junit4]   2> 83662 INFO  (SUITE-LargeVolumeBinaryJettyTest-seed#[2648413B1B80B622]-worker) [    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, loadOnStartup=true, schema=schema.xml, configSetProperties=configsetprops.json, transient=false, dataDir=data/}
   [junit4]   2> 83662 INFO  (SUITE-LargeVolumeBinaryJettyTest-seed#[2648413B1B80B622]-worker) [    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001/collection1
   [junit4]   2> 83663 INFO  (SUITE-LargeVolumeBinaryJettyTest-seed#[2648413B1B80B622]-worker) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 83664 INFO  (SUITE-LargeVolumeBinaryJettyTest-seed#[2648413B1B80B622]-worker) [    ] o.a.s.s.SolrDispatchFilter user.dir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1
   [junit4]   2> 83664 INFO  (SUITE-LargeVolumeBinaryJettyTest-seed#[2648413B1B80B622]-worker) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 83664 INFO  (coreLoadExecutor-152-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001/collection1'
   [junit4]   2> 83664 INFO  (coreLoadExecutor-152-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 83665 INFO  (coreLoadExecutor-152-thread-1) [    ] o.a.s.c.SolrResourceLoader using system property solr.solr.home: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001
   [junit4]   2> 83675 INFO  (coreLoadExecutor-152-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 83675 INFO  (coreLoadExecutor-152-thread-1) [    ] o.a.s.c.SolrConfig Adding specified lib dirs to ClassLoader
   [junit4]   2> 83675 WARN  (coreLoadExecutor-152-thread-1) [    ] o.a.s.c.SolrConfig Couldn't add files from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001/collection1/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001/collection1/../../../../contrib/extraction/lib
   [junit4]   2> 83675 WARN  (coreLoadExecutor-152-thread-1) [    ] o.a.s.c.SolrConfig Couldn't add files from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001/collection1/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001/collection1/../../../../dist
   [junit4]   2> 83676 WARN  (coreLoadExecutor-152-thread-1) [    ] o.a.s.c.SolrConfig Couldn't add files from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001/collection1/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001/collection1/../../../../contrib/clustering/lib
   [junit4]   2> 83676 WARN  (coreLoadExecutor-152-thread-1) [    ] o.a.s.c.SolrConfig Couldn't add files from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001/collection1/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001/collection1/../../../../dist
   [junit4]   2> 83676 WARN  (coreLoadExecutor-152-thread-1) [    ] o.a.s.c.SolrConfig Couldn't add files from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001/collection1/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001/collection1/../../../../contrib/langid/lib
   [junit4]   2> 83676 WARN  (coreLoadExecutor-152-thread-1) [    ] o.a.s.c.SolrConfig Couldn't add files from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001/collection1/../../../../dist filtered by solr-langid-\d.*\.jar to classpath: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001/collection1/../../../../dist
   [junit4]   2> 83676 WARN  (coreLoadExecutor-152-thread-1) [    ] o.a.s.c.SolrConfig Couldn't add files from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001/collection1/../../../../contrib/velocity/lib filtered by .*\.jar to classpath: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001/collection1/../../../../contrib/velocity/lib
   [junit4]   2> 83676 WARN  (coreLoadExecutor-152-thread-1) [    ] o.a.s.c.SolrConfig Couldn't add files from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001/collection1/../../../../dist filtered by solr-velocity-\d.*\.jar to classpath: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001/collection1/../../../../dist
   [junit4]   2> 83685 INFO  (coreLoadExecutor-152-thread-1) [    ] o.a.s.u.SolrIndexConfig IndexWriter infoStream solr logging is enabled
   [junit4]   2> 83689 INFO  (coreLoadExecutor-152-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 83718 INFO  (coreLoadExecutor-152-thread-1) [    ] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 83748 INFO  (coreLoadExecutor-152-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=example
   [junit4]   2> 84399 INFO  (coreLoadExecutor-152-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 84415 INFO  (coreLoadExecutor-152-thread-1) [    ] o.a.s.r.RestManager Registered ManagedResource impl org.apache.solr.rest.schema.analysis.ManagedWordSetResource for path /schema/analysis/stopwords/english
   [junit4]   2> 84415 INFO  (coreLoadExecutor-152-thread-1) [    ] o.a.s.r.RestManager Registered ManagedResource impl org.apache.solr.rest.schema.analysis.ManagedSynonymFilterFactory$SynonymManager for path /schema/analysis/synonyms/english
   [junit4]   2> 84416 INFO  (coreLoadExecutor-152-thread-1) [    ] o.a.s.r.RestManager Added observer of type org.apache.solr.rest.schema.analysis.ManagedStopFilterFactory to existing ManagedResource /schema/analysis/stopwords/english
   [junit4]   2> 84416 INFO  (coreLoadExecutor-152-thread-1) [    ] o.a.s.r.RestManager Added observer of type org.apache.solr.rest.schema.analysis.ManagedSynonymFilterFactory to existing ManagedResource /schema/analysis/synonyms/english
   [junit4]   2> 84420 INFO  (coreLoadExecutor-152-thread-1) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 84439 INFO  (coreLoadExecutor-152-thread-1) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 84480 INFO  (coreLoadExecutor-152-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001/collection1'
   [junit4]   2> 84481 INFO  (coreLoadExecutor-152-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001/collection1
   [junit4]   2> 84481 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 84481 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001/collection1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001/collection1/data/]
   [junit4]   2> 84481 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c4510ef
   [junit4]   2> 84481 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Added SolrEventListener for newSearcher: [org.apache.solr.core.QuerySenderListener{queries=[]}]
   [junit4]   2> 84481 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Added SolrEventListener for firstSearcher: [org.apache.solr.core.QuerySenderListener{queries=[{q=static firstSearcher warming in solrconfig.xml}]}]
   [junit4]   2> 84481 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.c.SchemaCodecFactory Using default compressionMode: BEST_SPEED
   [junit4]   2> 84482 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001/collection1/data
   [junit4]   2> 84482 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001/collection1/data/index/
   [junit4]   2> 84482 WARN  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 84482 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001/collection1/data/index
   [junit4]   2> 84482 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreLoadExecutor-152-thread-1]: init: current segments file is "segments"; deletionPolicy=org.apache.solr.core.IndexDeletionPolicyWrapper@60b73373
   [junit4]   2> 84482 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreLoadExecutor-152-thread-1]: delete []
   [junit4]   2> 84482 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreLoadExecutor-152-thread-1]: now checkpoint "" [0 segments ; isCommit = false]
   [junit4]   2> 84482 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreLoadExecutor-152-thread-1]: delete []
   [junit4]   2> 84482 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreLoadExecutor-152-thread-1]: 0 msec to checkpoint
   [junit4]   2> 84482 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-152-thread-1]: init: create=true
   [junit4]   2> 84482 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-152-thread-1]: 
   [junit4]   2> dir=MockDirectoryWrapper(RAMDirectory@7fb48b7b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2d81cb43)
   [junit4]   2> index=
   [junit4]   2> version=7.0.0
   [junit4]   2> analyzer=org.apache.solr.update.SolrIndexConfig$DelayedSchemaAnalyzer
   [junit4]   2> ramBufferSizeMB=100.0
   [junit4]   2> maxBufferedDocs=-1
   [junit4]   2> maxBufferedDeleteTerms=-1
   [junit4]   2> mergedSegmentWarmer=null
   [junit4]   2> delPolicy=org.apache.solr.core.IndexDeletionPolicyWrapper
   [junit4]   2> commit=null
   [junit4]   2> openMode=CREATE
   [junit4]   2> similarity=org.apache.solr.search.similarities.SchemaSimilarityFactory$1
   [junit4]   2> mergeScheduler=ConcurrentMergeScheduler: maxThreadCount=-1, maxMergeCount=-1, ioThrottle=true
   [junit4]   2> codec=Lucene62
   [junit4]   2> infoStream=org.apache.solr.update.LoggingInfoStream
   [junit4]   2> mergePolicy=[TieredMergePolicy: maxMergeAtOnce=10, maxMergeAtOnceExplicit=30, maxMergedSegmentMB=5120.0, floorSegmentMB=2.0, forceMergeDeletesPctAllowed=10.0, segmentsPerTier=10.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1
   [junit4]   2> indexerThreadPool=org.apache.lucene.index.DocumentsWriterPerThreadPool@54ff4cf9
   [junit4]   2> readerPooling=false
   [junit4]   2> perThreadHardLimitMB=1945
   [junit4]   2> useCompoundFile=false
   [junit4]   2> commitOnClose=true
   [junit4]   2> indexSort=null
   [junit4]   2> writer=org.apache.solr.update.SolrIndexWriter@fcbfa4e
   [junit4]   2> 
   [junit4]   2> 84483 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-152-thread-1]: MMapDirectory.UNMAP_SUPPORTED=true
   [junit4]   2> 84483 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-152-thread-1]: now flush at close
   [junit4]   2> 84483 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-152-thread-1]:   start flush: applyAllDeletes=true
   [junit4]   2> 84483 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-152-thread-1]:   index before flush 
   [junit4]   2> 84483 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][coreLoadExecutor-152-thread-1]: startFullFlush
   [junit4]   2> 84483 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][coreLoadExecutor-152-thread-1]: coreLoadExecutor-152-thread-1 finishFullFlush success=true
   [junit4]   2> 84483 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-152-thread-1]: apply all deletes during flush
   [junit4]   2> 84483 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-152-thread-1]: now apply all deletes for all segments maxDoc=0
   [junit4]   2> 84483 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [BD][coreLoadExecutor-152-thread-1]: prune sis=segments:  minGen=9223372036854775807 packetCount=0
   [junit4]   2> 84483 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [MS][coreLoadExecutor-152-thread-1]: initDynamicDefaults spins=false maxThreadCount=1 maxMergeCount=6
   [junit4]   2> 84483 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [MS][coreLoadExecutor-152-thread-1]: updateMergeThreads ioThrottle=true targetMBPerSec=10240.0 MB/sec
   [junit4]   2> 84483 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [MS][coreLoadExecutor-152-thread-1]: now merge
   [junit4]   2> 84483 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [MS][coreLoadExecutor-152-thread-1]:   index: 
   [junit4]   2> 84483 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [MS][coreLoadExecutor-152-thread-1]:   no more merges pending; now return
   [junit4]   2> 84484 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-152-thread-1]: waitForMerges
   [junit4]   2> 84484 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-152-thread-1]: waitForMerges done
   [junit4]   2> 84484 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-152-thread-1]: commit: start
   [junit4]   2> 84484 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-152-thread-1]: commit: enter lock
   [junit4]   2> 84484 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-152-thread-1]: commit: now prepare
   [junit4]   2> 84484 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-152-thread-1]: prepareCommit: flush
   [junit4]   2> 84484 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-152-thread-1]:   index before flush 
   [junit4]   2> 84484 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][coreLoadExecutor-152-thread-1]: startFullFlush
   [junit4]   2> 84484 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-152-thread-1]: apply all deletes during flush
   [junit4]   2> 84484 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-152-thread-1]: now apply all deletes for all segments maxDoc=0
   [junit4]   2> 84484 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [BD][coreLoadExecutor-152-thread-1]: prune sis=segments:  minGen=9223372036854775807 packetCount=0
   [junit4]   2> 84484 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][coreLoadExecutor-152-thread-1]: coreLoadExecutor-152-thread-1 finishFullFlush success=true
   [junit4]   2> 84484 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-152-thread-1]: startCommit(): start
   [junit4]   2> 84484 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-152-thread-1]: startCommit index= changeCount=2
   [junit4]   2> 84484 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-152-thread-1]: startCommit: wrote pending segments file "pending_segments_1"
   [junit4]   2> 84484 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-152-thread-1]: done all syncs: []
   [junit4]   2> 84484 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-152-thread-1]: commit: pendingCommit != null
   [junit4]   2> 84484 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-152-thread-1]: commit: done writing segments file "segments_1"
   [junit4]   2> 84484 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreLoadExecutor-152-thread-1]: now checkpoint "" [0 segments ; isCommit = true]
   [junit4]   2> 84484 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@7fb48b7b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2d81cb43),segFN=segments_1,generation=1}
   [junit4]   2> 84484 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 84484 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreLoadExecutor-152-thread-1]: 0 msec to checkpoint
   [junit4]   2> 84484 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreLoadExecutor-152-thread-1]: delete []
   [junit4]   2> 84484 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-152-thread-1]: commit: took 0.5 msec
   [junit4]   2> 84484 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-152-thread-1]: commit: done
   [junit4]   2> 84484 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-152-thread-1]: rollback
   [junit4]   2> 84484 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-152-thread-1]: all running merges have aborted
   [junit4]   2> 84484 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-152-thread-1]: rollback: done finish merges
   [junit4]   2> 84484 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][coreLoadExecutor-152-thread-1]: abort
   [junit4]   2> 84484 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][coreLoadExecutor-152-thread-1]: done abort success=true
   [junit4]   2> 84484 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-152-thread-1]: rollback: infos=
   [junit4]   2> 84484 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreLoadExecutor-152-thread-1]: now checkpoint "" [0 segments ; isCommit = false]
   [junit4]   2> 84484 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreLoadExecutor-152-thread-1]: delete []
   [junit4]   2> 84484 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreLoadExecutor-152-thread-1]: 0 msec to checkpoint
   [junit4]   2> 84485 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreLoadExecutor-152-thread-1]: delete []
   [junit4]   2> 84490 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001/collection1/data/
   [junit4]   2> 84519 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.DirectoryFactory /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 84520 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.c.PluginBag queryResponseWriter : 'velocity' created with startup=lazy 
   [junit4]   2> 84520 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.r.XSLTResponseWriter xsltCacheLifetimeSeconds=5
   [junit4]   2> 84527 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 84532 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 84533 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 84534 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 84535 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 84538 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : '/update/extract' created with startup=lazy 
   [junit4]   2> 84538 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : '/analysis/field' created with startup=lazy 
   [junit4]   2> 84538 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : '/analysis/document' created with startup=lazy 
   [junit4]   2> 84538 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : '/spell' created with startup=lazy 
   [junit4]   2> 84538 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : '/suggest' created with startup=lazy 
   [junit4]   2> 84538 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : '/tvrh' created with startup=lazy 
   [junit4]   2> 84538 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : '/terms' created with startup=lazy 
   [junit4]   2> 84538 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : '/elevate' created with startup=lazy 
   [junit4]   2> 84538 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,/browse,/update/json/docs,/admin/luke,/export,/get,/admin/properties,/elevate,/update/json,/admin/threads,/query,/analysis/field,/analysis/document,/spell,/update/csv,/sql,/graph,/tvrh,/select,/admin/segments,/admin/system,/suggest,/replication,/config,/stream,/schema,/admin/plugins,/admin/logging,/admin/ping,/update,/admin/file,/debug/dump,/terms,/update/extract
   [junit4]   2> 84539 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 84539 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 84539 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 84541 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 15000ms; 
   [junit4]   2> 84541 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 84541 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreLoadExecutor-152-thread-1]: init: current segments file is "segments_1"; deletionPolicy=org.apache.solr.core.IndexDeletionPolicyWrapper@60b73373
   [junit4]   2> 84541 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreLoadExecutor-152-thread-1]: init: load commit "segments_1"
   [junit4]   2> 84541 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreLoadExecutor-152-thread-1]: delete []
   [junit4]   2> 84542 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@7fb48b7b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2d81cb43),segFN=segments_1,generation=1}
   [junit4]   2> 84542 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 84542 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreLoadExecutor-152-thread-1]: now checkpoint "" [0 segments ; isCommit = false]
   [junit4]   2> 84542 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreLoadExecutor-152-thread-1]: delete []
   [junit4]   2> 84542 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreLoadExecutor-152-thread-1]: 0 msec to checkpoint
   [junit4]   2> 84542 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-152-thread-1]: init: create=false
   [junit4]   2> 84542 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-152-thread-1]: 
   [junit4]   2> dir=MockDirectoryWrapper(RAMDirectory@7fb48b7b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2d81cb43)
   [junit4]   2> index=
   [junit4]   2> version=7.0.0
   [junit4]   2> analyzer=org.apache.solr.update.SolrIndexConfig$DelayedSchemaAnalyzer
   [junit4]   2> ramBufferSizeMB=100.0
   [junit4]   2> maxBufferedDocs=-1
   [junit4]   2> maxBufferedDeleteTerms=-1
   [junit4]   2> mergedSegmentWarmer=null
   [junit4]   2> delPolicy=org.apache.solr.core.IndexDeletionPolicyWrapper
   [junit4]   2> commit=null
   [junit4]   2> openMode=APPEND
   [junit4]   2> similarity=org.apache.solr.search.similarities.SchemaSimilarityFactory$1
   [junit4]   2> mergeScheduler=ConcurrentMergeScheduler: maxThreadCount=-1, maxMergeCount=-1, ioThrottle=true
   [junit4]   2> codec=Lucene62
   [junit4]   2> infoStream=org.apache.solr.update.LoggingInfoStream
   [junit4]   2> mergePolicy=[TieredMergePolicy: maxMergeAtOnce=10, maxMergeAtOnceExplicit=30, maxMergedSegmentMB=5120.0, floorSegmentMB=2.0, forceMergeDeletesPctAllowed=10.0, segmentsPerTier=10.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1
   [junit4]   2> indexerThreadPool=org.apache.lucene.index.DocumentsWriterPerThreadPool@c337e79
   [junit4]   2> readerPooling=false
   [junit4]   2> perThreadHardLimitMB=1945
   [junit4]   2> useCompoundFile=false
   [junit4]   2> commitOnClose=true
   [junit4]   2> indexSort=null
   [junit4]   2> writer=org.apache.solr.update.SolrIndexWriter@77390e66
   [junit4]   2> 
   [junit4]   2> 84542 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-152-thread-1]: MMapDirectory.UNMAP_SUPPORTED=true
   [junit4]   2> 84542 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-152-thread-1]: flush at getReader
   [junit4]   2> 84542 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][coreLoadExecutor-152-thread-1]: startFullFlush
   [junit4]   2> 84542 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-152-thread-1]: apply all deletes during flush
   [junit4]   2> 84542 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-152-thread-1]: now apply all deletes for all segments maxDoc=0
   [junit4]   2> 84542 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [BD][coreLoadExecutor-152-thread-1]: prune sis=segments_1:  minGen=9223372036854775807 packetCount=0
   [junit4]   2> 84542 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-152-thread-1]: incRefDeleter for NRT reader version=2 segments=
   [junit4]   2> 84542 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-152-thread-1]: return reader version=2 reader=StandardDirectoryReader(segments_1:2:nrt)
   [junit4]   2> 84542 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][coreLoadExecutor-152-thread-1]: coreLoadExecutor-152-thread-1 finishFullFlush success=true
   [junit4]   2> 84542 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][coreLoadExecutor-152-thread-1]: getReader took 0 msec
   [junit4]   2> 84542 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@40360baa[collection1] main]
   [junit4]   2> 84542 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001/collection1/conf
   [junit4]   2> 84542 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001/collection1/conf}
   [junit4]   2> 84542 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001/collection1/conf
   [junit4]   2> 84543 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Loaded LinkedHashMap at path _rest_managed.json using file:dir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001/collection1/conf
   [junit4]   2> 84543 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.r.ManagedResource Loaded initArgs {} for /rest/managed
   [junit4]   2> 84543 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 2 registered ManagedResources
   [junit4]   2> 84543 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _schema_analysis_stopwords_english.json using file:dir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001/collection1/conf
   [junit4]   2> 84543 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Loaded LinkedHashMap at path _schema_analysis_stopwords_english.json using file:dir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001/collection1/conf
   [junit4]   2> 84543 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.r.ManagedResource Loaded initArgs {ignoreCase=true} for /schema/analysis/stopwords/english
   [junit4]   2> 84543 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.r.s.a.ManagedWordSetResource Loaded 35 words for /schema/analysis/stopwords/english
   [junit4]   2> 84544 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.r.ManagedResource Notified 2 observers of /schema/analysis/stopwords/english
   [junit4]   2> 84544 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _schema_analysis_synonyms_english.json using file:dir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001/collection1/conf
   [junit4]   2> 84545 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Loaded LinkedHashMap at path _schema_analysis_synonyms_english.json using file:dir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001/collection1/conf
   [junit4]   2> 84545 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.r.ManagedResource Loaded initArgs {ignoreCase=true,format=solr} for /schema/analysis/synonyms/english
   [junit4]   2> 84545 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.r.s.a.ManagedSynonymFilterFactory Loaded 3 synonym mappings for /schema/analysis/synonyms/english
   [junit4]   2> 84545 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.r.ManagedResource Notified 2 observers of /schema/analysis/synonyms/english
   [junit4]   2> 84545 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 84545 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=default,field=text,classname=solr.DirectSolrSpellChecker,distanceMeasure=internal,accuracy=0.5,maxEdits=2,minPrefix=1,maxInspections=5,minQueryLength=4,maxQueryFrequency=0.01}
   [junit4]   2> 84545 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.h.c.SpellCheckComponent No queryConverter defined, using default converter
   [junit4]   2> 84547 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.h.c.SuggestComponent Initializing SuggestComponent
   [junit4]   2> 84547 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.s.s.SolrSuggester init: {name=mySuggester,lookupImpl=FuzzyLookupFactory,dictionaryImpl=DocumentDictionaryFactory,field=cat,weightField=price,suggestAnalyzerFieldType=string,buildOnStartup=false}
   [junit4]   2> 84547 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.s.s.SolrSuggester Dictionary loaded with params: {name=mySuggester,lookupImpl=FuzzyLookupFactory,dictionaryImpl=DocumentDictionaryFactory,field=cat,weightField=price,suggestAnalyzerFieldType=string,buildOnStartup=false}
   [junit4]   2> 84547 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.h.c.QueryElevationComponent Loading QueryElevation from: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001/collection1/conf/elevate.xml
   [junit4]   2> 84551 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 84551 INFO  (searcherExecutor-153-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.QuerySenderListener QuerySenderListener sending requests to Searcher@40360baa[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 84552 INFO  (searcherExecutor-153-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=static+firstSearcher+warming+in+solrconfig.xml&distrib=false&event=firstSearcher} hits=0 status=0 QTime=0
   [junit4]   2> 84552 INFO  (searcherExecutor-153-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.QuerySenderListener QuerySenderListener done.
   [junit4]   2> 84552 INFO  (searcherExecutor-153-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 84552 INFO  (searcherExecutor-153-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 84552 INFO  (searcherExecutor-153-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@40360baa[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 84554 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 84554 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 84555 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 84555 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1539028335087058944
   [junit4]   2> 84556 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.u.UpdateLog Took 1.0ms to seed version buckets with highest version 1539028335087058944
   [junit4]   2> 84556 INFO  (coreLoadExecutor-152-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 84666 INFO  (SUITE-LargeVolumeBinaryJettyTest-seed#[2648413B1B80B622]-worker) [    ] o.a.s.SolrJettyTestBase Jetty Assigned Port#38580
   [junit4]   2> 84707 INFO  (TEST-LargeVolumeBinaryJettyTest.testMultiThreaded-seed#[2648413B1B80B622]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testMultiThreaded
   [junit4]   2> 84786 INFO  (qtp1728437887-881) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][qtp1728437887-881]: anyChanges? numDocsInRam=0 deletes=true hasTickets:false pendingChangesInFullFlush: false
   [junit4]   2> 84786 INFO  (qtp1728437887-881) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][qtp1728437887-881]: anyChanges? numDocsInRam=0 deletes=true hasTickets:false pendingChangesInFullFlush: false
   [junit4]   2> 84786 INFO  (qtp1728437887-881) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-881]: nrtIsCurrent: infoVersion matches: true; DW changes: true; BD changes: false
   [junit4]   2> 84786 INFO  (qtp1728437887-881) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-881]: flush at getReader
   [junit4]   2> 84786 INFO  (qtp1728437887-881) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][qtp1728437887-881]: startFullFlush
   [junit4]   2> 84786 INFO  (qtp1728437887-881) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][qtp1728437887-881]: anyChanges? numDocsInRam=0 deletes=true hasTickets:false pendingChangesInFullFlush: false
   [junit4]   2> 84786 INFO  (qtp1728437887-881) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][qtp1728437887-881]: qtp1728437887-881: flush naked frozen global deletes
   [junit4]   2> 84787 INFO  (qtp1728437887-881) [    x:collection1] o.a.s.u.LoggingInfoStream [BD][qtp1728437887-881]: push deletes  1 deleted queries bytesUsed=100 segmentPrivate?=false delGen=2 packetCount=1 totBytesUsed=100
   [junit4]   2> 84787 INFO  (qtp1728437887-881) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][qtp1728437887-881]: flush: push buffered updates:  1 deleted queries bytesUsed=100
   [junit4]   2> 84787 INFO  (qtp1728437887-881) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-881]: apply all deletes during flush
   [junit4]   2> 84787 INFO  (qtp1728437887-881) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-881]: now apply all deletes for all segments maxDoc=0
   [junit4]   2> 84787 INFO  (qtp1728437887-881) [    x:collection1] o.a.s.u.LoggingInfoStream [BD][qtp1728437887-881]: prune sis=segments_1:  minGen=9223372036854775807 packetCount=1
   [junit4]   2> 84787 INFO  (qtp1728437887-881) [    x:collection1] o.a.s.u.LoggingInfoStream [BD][qtp1728437887-881]: pruneDeletes: prune 1 packets; 0 packets remain
   [junit4]   2> 84787 INFO  (qtp1728437887-881) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-881]: incRefDeleter for NRT reader version=2 segments=
   [junit4]   2> 84787 INFO  (qtp1728437887-881) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-881]: return reader version=2 reader=StandardDirectoryReader(segments_1:2:nrt)
   [junit4]   2> 84787 INFO  (qtp1728437887-881) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][qtp1728437887-881]: qtp1728437887-881 finishFullFlush success=true
   [junit4]   2> 84787 INFO  (qtp1728437887-881) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-881]: getReader took 1 msec
   [junit4]   2> 84787 INFO  (qtp1728437887-881) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][qtp1728437887-881]: delete []
   [junit4]   2> 84787 INFO  (qtp1728437887-881) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-881]: decRefDeleter for NRT reader version=2 segments=
   [junit4]   2> 84788 INFO  (qtp1728437887-881) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{deleteByQuery=*:* (-1539028335255879680)} 0 71
   [junit4]   2> 84790 INFO  (TEST-LargeVolumeBinaryJettyTest.testMultiThreaded-seed#[2648413B1B80B622]) [    ] o.a.s.c.s.LargeVolumeTestBase Started thread: 0
   [junit4]   2> 84792 INFO  (TEST-LargeVolumeBinaryJettyTest.testMultiThreaded-seed#[2648413B1B80B622]) [    ] o.a.s.c.s.LargeVolumeTestBase Started thread: 1
   [junit4]   2> 84793 INFO  (TEST-LargeVolumeBinaryJettyTest.testMultiThreaded-seed#[2648413B1B80B622]) [    ] o.a.s.c.s.LargeVolumeTestBase Started thread: 2
   [junit4]   2> 84800 INFO  (TEST-LargeVolumeBinaryJettyTest.testMultiThreaded-seed#[2648413B1B80B622]) [    ] o.a.s.c.s.LargeVolumeTestBase Started thread: 3
   [junit4]   2> 84801 INFO  (TEST-LargeVolumeBinaryJettyTest.testMultiThreaded-seed#[2648413B1B80B622]) [    ] o.a.s.c.s.LargeVolumeTestBase Started thread: 4
   [junit4]   2> 84848 INFO  (qtp1728437887-883) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[T1:0 (1539028335340814336), T1:1 (1539028335385903109), T1:2 (1539028335385903111), T1:3 (1539028335385903114), T1:4 (1539028335385903116), T1:5 (1539028335385903118), T1:6 (1539028335385903120), T1:7 (1539028335385903126), T1:8 (1539028335385903137), T1:9 (1539028335385903140), ... (100 adds)]} 0 51
   [junit4]   2> 84848 INFO  (qtp1728437887-882) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[T0:0 (1539028335339765760), T0:1 (1539028335385903104), T0:2 (1539028335385903105), T0:3 (1539028335385903107), T0:4 (1539028335385903108), T0:5 (1539028335385903110), T0:6 (1539028335385903112), T0:7 (1539028335385903113), T0:8 (1539028335385903115), T0:9 (1539028335385903117), ... (100 adds)]} 0 52
   [junit4]   2> 84855 INFO  (qtp1728437887-884) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[T2:0 (1539028335340814337), T2:1 (1539028335385903106), T2:2 (1539028335385903130), T2:3 (1539028335385903133), T2:4 (1539028335385903135), T2:5 (1539028335385903138), T2:6 (1539028335385903141), T2:7 (1539028335386951681), T2:8 (1539028335386951684), T2:9 (1539028335386951687), ... (100 adds)]} 0 58
   [junit4]   2> 84864 INFO  (qtp1728437887-878) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[T3:0 (1539028335347105792), T3:1 (1539028335389048864), T3:2 (1539028335390097421), T3:3 (1539028335390097422), T3:4 (1539028335390097423), T3:5 (1539028335390097424), T3:6 (1539028335390097425), T3:7 (1539028335390097426), T3:8 (1539028335390097427), T3:9 (1539028335390097428), ... (100 adds)]} 0 61
   [junit4]   2> 84865 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 84865 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: commit: start
   [junit4]   2> 84865 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: commit: enter lock
   [junit4]   2> 84865 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: commit: now prepare
   [junit4]   2> 84865 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: prepareCommit: flush
   [junit4]   2> 84865 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]:   index before flush 
   [junit4]   2> 84865 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][qtp1728437887-877]: startFullFlush
   [junit4]   2> 84865 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][qtp1728437887-877]: anyChanges? numDocsInRam=400 deletes=true hasTickets:false pendingChangesInFullFlush: false
   [junit4]   2> 84865 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [DWFC][qtp1728437887-877]: addFlushableState DocumentsWriterPerThread [pendingDeletes=gen=0 306 deleted terms (unique count=306) bytesUsed=71259, segment=_0, aborted=false, numDocsInRAM=158, deleteQueue=DWDQ: [ generation: 2 ]]
   [junit4]   2> 84865 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [DWFC][qtp1728437887-877]: addFlushableState DocumentsWriterPerThread [pendingDeletes=gen=0 396 deleted terms (unique count=396) bytesUsed=92232, segment=_1, aborted=false, numDocsInRAM=154, deleteQueue=DWDQ: [ generation: 2 ]]
   [junit4]   2> 84865 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [DWFC][qtp1728437887-877]: addFlushableState DocumentsWriterPerThread [pendingDeletes=gen=0 300 deleted terms (unique count=300) bytesUsed=69865, segment=_2, aborted=false, numDocsInRAM=88, deleteQueue=DWDQ: [ generation: 2 ]]
   [junit4]   2> 84870 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [DWPT][qtp1728437887-877]: flush postings as segment _0 numDocs=158
   [junit4]   2> 84870 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: 0 msec to write norms
   [junit4]   2> 84870 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: 0 msec to write docValues
   [junit4]   2> 84870 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: 0 msec to write points
   [junit4]   2> 84870 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: 0 msec to finish stored fields
   [junit4]   2> 84873 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: 2 msec to write postings and finish vectors
   [junit4]   2> 84874 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: 0 msec to write fieldInfos
   [junit4]   2> 84874 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [DWPT][qtp1728437887-877]: new segment has 0 deleted docs
   [junit4]   2> 84874 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [DWPT][qtp1728437887-877]: new segment has no vectors; norms; docValues; prox; freqs
   [junit4]   2> 84874 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [DWPT][qtp1728437887-877]: flushedFiles=[_0_Lucene54_0.dvm, _0_Lucene50_0.doc, _0_Lucene50_0.tim, _0_Lucene50_0.pos, _0.nvd, _0.fdx, _0_Lucene54_0.dvd, _0_Lucene50_0.tip, _0.fdt, _0.nvm, _0.fnm]
   [junit4]   2> 84874 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [DWPT][qtp1728437887-877]: flushed codec=Lucene62
   [junit4]   2> 84874 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [DWPT][qtp1728437887-877]: flushed: segment=_0 ramUsed=0.203 MB newFlushedSize=0.006 MB docs/MB=26,558.995
   [junit4]   2> 84874 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [DWPT][qtp1728437887-877]: flush time 4.15787 msec
   [junit4]   2> 84874 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [DWPT][qtp1728437887-877]: flush postings as segment _1 numDocs=154
   [junit4]   2> 84874 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: 0 msec to write norms
   [junit4]   2> 84874 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: 0 msec to write docValues
   [junit4]   2> 84874 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: 0 msec to write points
   [junit4]   2> 84874 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: 0 msec to finish stored fields
   [junit4]   2> 84878 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: 3 msec to write postings and finish vectors
   [junit4]   2> 84878 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: 0 msec to write fieldInfos
   [junit4]   2> 84878 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [DWPT][qtp1728437887-877]: new segment has 0 deleted docs
   [junit4]   2> 84878 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [DWPT][qtp1728437887-877]: new segment has no vectors; norms; docValues; prox; freqs
   [junit4]   2> 84878 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [DWPT][qtp1728437887-877]: flushedFiles=[_1_Lucene50_0.doc, _1_Lucene50_0.tim, _1_Lucene54_0.dvd, _1.nvd, _1_Lucene50_0.pos, _1.fdx, _1.nvm, _1_Lucene54_0.dvm, _1.fnm, _1.fdt, _1_Lucene50_0.tip]
   [junit4]   2> 84878 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [DWPT][qtp1728437887-877]: flushed codec=Lucene62
   [junit4]   2> 84878 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [DWPT][qtp1728437887-877]: flushed: segment=_1 ramUsed=0.202 MB newFlushedSize=0.006 MB docs/MB=26,154.957
   [junit4]   2> 84879 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [DWPT][qtp1728437887-877]: flush time 4.699413 msec
   [junit4]   2> 84879 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [DWPT][qtp1728437887-877]: flush postings as segment _2 numDocs=88
   [junit4]   2> 84879 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: 0 msec to write norms
   [junit4]   2> 84879 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: 0 msec to write docValues
   [junit4]   2> 84879 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: 0 msec to write points
   [junit4]   2> 84879 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: 0 msec to finish stored fields
   [junit4]   2> 84881 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: 2 msec to write postings and finish vectors
   [junit4]   2> 84881 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: 0 msec to write fieldInfos
   [junit4]   2> 84881 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [DWPT][qtp1728437887-877]: new segment has 0 deleted docs
   [junit4]   2> 84881 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [DWPT][qtp1728437887-877]: new segment has no vectors; norms; docValues; prox; freqs
   [junit4]   2> 84881 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [DWPT][qtp1728437887-877]: flushedFiles=[_2_Lucene54_0.dvd, _2.nvd, _2.fdt, _2.nvm, _2_Lucene54_0.dvm, _2_Lucene50_0.pos, _2.fnm, _2.fdx, _2_Lucene50_0.doc, _2_Lucene50_0.tim, _2_Lucene50_0.tip]
   [junit4]   2> 84881 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [DWPT][qtp1728437887-877]: flushed codec=Lucene62
   [junit4]   2> 84881 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [DWPT][qtp1728437887-877]: flushed: segment=_2 ramUsed=0.197 MB newFlushedSize=0.004 MB docs/MB=20,410.238
   [junit4]   2> 84882 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [DWPT][qtp1728437887-877]: flush time 2.86594 msec
   [junit4]   2> 84882 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][qtp1728437887-877]: publishFlushedSegment seg-private updates=null
   [junit4]   2> 84882 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: publishFlushedSegment
   [junit4]   2> 84882 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [BD][qtp1728437887-877]: push deletes  400 deleted terms (unique count=400) bytesUsed=2112 segmentPrivate?=false delGen=4 packetCount=1 totBytesUsed=2112
   [junit4]   2> 84882 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: publish sets newSegment delGen=5 seg=_0(7.0.0):C158
   [junit4]   2> 84882 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][qtp1728437887-877]: now checkpoint "_0(7.0.0):C158" [1 segments ; isCommit = false]
   [junit4]   2> 84882 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][qtp1728437887-877]: delete []
   [junit4]   2> 84882 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][qtp1728437887-877]: 0 msec to checkpoint
   [junit4]   2> 84882 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][qtp1728437887-877]: publishFlushedSegment seg-private updates=null
   [junit4]   2> 84882 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: publishFlushedSegment
   [junit4]   2> 84882 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: publish sets newSegment delGen=6 seg=_1(7.0.0):C154
   [junit4]   2> 84882 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][qtp1728437887-877]: now checkpoint "_0(7.0.0):C158 _1(7.0.0):C154" [2 segments ; isCommit = false]
   [junit4]   2> 84882 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][qtp1728437887-877]: delete []
   [junit4]   2> 84882 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][qtp1728437887-877]: 0 msec to checkpoint
   [junit4]   2> 84882 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][qtp1728437887-877]: publishFlushedSegment seg-private updates=null
   [junit4]   2> 84882 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: publishFlushedSegment
   [junit4]   2> 84882 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: publish sets newSegment delGen=7 seg=_2(7.0.0):C88
   [junit4]   2> 84882 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][qtp1728437887-877]: now checkpoint "_0(7.0.0):C158 _1(7.0.0):C154 _2(7.0.0):C88" [3 segments ; isCommit = false]
   [junit4]   2> 84882 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][qtp1728437887-877]: delete []
   [junit4]   2> 84882 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][qtp1728437887-877]: 0 msec to checkpoint
   [junit4]   2> 84882 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: apply all deletes during flush
   [junit4]   2> 84882 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: now apply all deletes for all segments maxDoc=400
   [junit4]   2> 84882 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [BD][qtp1728437887-877]: applyDeletes: open segment readers took 0 msec
   [junit4]   2> 84882 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [BD][qtp1728437887-877]: applyDeletes: infos=[_0(7.0.0):C158, _1(7.0.0):C154, _2(7.0.0):C88] packetCount=1
   [junit4]   2> 84882 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [BD][qtp1728437887-877]: applyDeletes took 0 msec for 3 segments, 0 newly deleted docs (query deletes), 0 visited terms, allDeleted=null
   [junit4]   2> 84882 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [BD][qtp1728437887-877]: prune sis=segments_1: _0(7.0.0):C158 _1(7.0.0):C154 _2(7.0.0):C88 minGen=5 packetCount=1
   [junit4]   2> 84882 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [BD][qtp1728437887-877]: pruneDeletes: prune 1 packets; 0 packets remain
   [junit4]   2> 84882 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][qtp1728437887-877]: qtp1728437887-877 finishFullFlush success=true
   [junit4]   2> 84882 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [TMP][qtp1728437887-877]: findMerges: 3 segments
   [junit4]   2> 84882 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [TMP][qtp1728437887-877]:   seg=_0(7.0.0):C158 size=0.006 MB [floored]
   [junit4]   2> 84882 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [TMP][qtp1728437887-877]:   seg=_1(7.0.0):C154 size=0.006 MB [floored]
   [junit4]   2> 84882 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [TMP][qtp1728437887-877]:   seg=_2(7.0.0):C88 size=0.004 MB [floored]
   [junit4]   2> 84882 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [TMP][qtp1728437887-877]:   allowedSegmentCount=1 vs count=3 (eligible count=3) tooBigCount=0
   [junit4]   2> 84883 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [MS][qtp1728437887-877]: initDynamicDefaults spins=false maxThreadCount=1 maxMergeCount=6
   [junit4]   2> 84883 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [MS][qtp1728437887-877]: now merge
   [junit4]   2> 84883 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [MS][qtp1728437887-877]:   index: _0(7.0.0):C158 _1(7.0.0):C154 _2(7.0.0):C88
   [junit4]   2> 84883 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [MS][qtp1728437887-877]:   no more merges pending; now return
   [junit4]   2> 84883 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: startCommit(): start
   [junit4]   2> 84883 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: startCommit index=_0(7.0.0):C158 _1(7.0.0):C154 _2(7.0.0):C88 changeCount=8
   [junit4]   2> 84883 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: startCommit: wrote pending segments file "pending_segments_2"
   [junit4]   2> 84883 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: done all syncs: [_0_Lucene50_0.doc, _1.si, _1.nvd, _0.nvd, _2.fdt, _2_Lucene50_0.pos, _1.nvm, _1.fnm, _2.fdx, _2_Lucene50_0.tim, _2_Lucene50_0.tip, _1_Lucene50_0.tim, _2.nvd, _1_Lucene50_0.pos, _2.nvm, _2.fnm, _1.fdx, _1.fdt, _1_Lucene50_0.tip, _2_Lucene54_0.dvd, _0.si, _0_Lucene50_0.tim, _2.si, _0_Lucene50_0.pos, _2_Lucene54_0.dvm, _0.fdx, _0_Lucene50_0.tip, _2_Lucene50_0.doc, _0.fdt, _1_Lucene50_0.doc, _0_Lucene54_0.dvm, _1_Lucene54_0.dvd, _0_Lucene54_0.dvd, _1_Lucene54_0.dvm, _0.nvm, _0.fnm]
   [junit4]   2> 84883 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: commit: pendingCommit != null
   [junit4]   2> 84883 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: commit: done writing segments file "segments_2"
   [junit4]   2> 84883 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][qtp1728437887-877]: now checkpoint "_0(7.0.0):C158 _1(7.0.0):C154 _2(7.0.0):C88" [3 segments ; isCommit = true]
   [junit4]   2> 84883 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@7fb48b7b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2d81cb43),segFN=segments_1,generation=1}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@7fb48b7b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2d81cb43),segFN=segments_2,generation=2}
   [junit4]   2> 84885 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 2
   [junit4]   2> 84885 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][qtp1728437887-877]: deleteCommits: now decRef commit "segments_1"
   [junit4]   2> 84885 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][qtp1728437887-877]: delete [segments_1]
   [junit4]   2> 84885 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][qtp1728437887-877]: 1 msec to checkpoint
   [junit4]   2> 84885 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][qtp1728437887-877]: delete []
   [junit4]   2> 84885 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: commit: took 20.0 msec
   [junit4]   2> 84885 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: commit: done
   [junit4]   2> 84887 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: nrtIsCurrent: infoVersion matches: false; DW changes: false; BD changes: false
   [junit4]   2> 84887 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: flush at getReader
   [junit4]   2> 84887 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][qtp1728437887-877]: startFullFlush
   [junit4]   2> 84887 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: apply all deletes during flush
   [junit4]   2> 84887 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: now apply all deletes for all segments maxDoc=400
   [junit4]   2> 84887 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [BD][qtp1728437887-877]: applyDeletes: open segment readers took 0 msec
   [junit4]   2> 84887 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [BD][qtp1728437887-877]: applyDeletes: no segments; skipping
   [junit4]   2> 84887 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [BD][qtp1728437887-877]: prune sis=segments_2: _0(7.0.0):C158 _1(7.0.0):C154 _2(7.0.0):C88 minGen=5 packetCount=0
   [junit4]   2> 84888 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: incRefDeleter for NRT reader version=10 segments=_0(7.0.0):C158 _1(7.0.0):C154 _2(7.0.0):C88
   [junit4]   2> 84888 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: return reader version=10 reader=StandardDirectoryReader(segments_2:10:nrt _0(7.0.0):C158 _1(7.0.0):C154 _2(7.0.0):C88)
   [junit4]   2> 84888 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][qtp1728437887-877]: qtp1728437887-877 finishFullFlush success=true
   [junit4]   2> 84888 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-877]: getReader took 1 msec
   [junit4]   2> 84888 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@342696c9[collection1] main]
   [junit4]   2> 84888 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 84888 INFO  (searcherExecutor-153-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.QuerySenderListener QuerySenderListener sending requests to Searcher@342696c9[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C158) Uninverting(_1(7.0.0):C154) Uninverting(_2(7.0.0):C88)))}
   [junit4]   2> 84888 INFO  (searcherExecutor-153-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.QuerySenderListener QuerySenderListener done.
   [junit4]   2> 84889 INFO  (searcherExecutor-153-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@342696c9[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C158) Uninverting(_1(7.0.0):C154) Uninverting(_2(7.0.0):C88)))}
   [junit4]   2> 84889 INFO  (searcherExecutor-153-thread-1-processing-x:collection1) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][searcherExecutor-153-thread-1-processing-x:collection1]: delete []
   [junit4]   2> 84889 INFO  (searcherExecutor-153-thread-1-processing-x:collection1) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][searcherExecutor-153-thread-1-processing-x:collection1]: decRefDeleter for NRT reader version=2 segments=
   [junit4]   2> 84889 INFO  (qtp1728437887-877) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 24
   [junit4]   2> 84890 INFO  (qtp1728437887-882) [    x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=true,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 84891 WARN  (qtp1728437887-882) [    x:collection1] o.a.s.u.DirectUpdateHandler2 Starting optimize... Reading and rewriting the entire index! Use with care.
   [junit4]   2> 84891 INFO  (qtp1728437887-882) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-882]: forceMerge: index now _0(7.0.0):C158 _1(7.0.0):C154 _2(7.0.0):C88
   [junit4]   2> 84891 INFO  (qtp1728437887-882) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-882]: now flush at forceMerge
   [junit4]   2> 84891 INFO  (qtp1728437887-882) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-882]:   start flush: applyAllDeletes=true
   [junit4]   2> 84892 INFO  (qtp1728437887-882) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-882]:   index before flush _0(7.0.0):C158 _1(7.0.0):C154 _2(7.0.0):C88
   [junit4]   2> 84892 INFO  (qtp1728437887-882) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][qtp1728437887-882]: startFullFlush
   [junit4]   2> 84892 INFO  (qtp1728437887-882) [    x:collection1] o.a.s.u.LoggingInfoStream [DW][qtp1728437887-882]: qtp1728437887-882 finishFullFlush success=true
   [junit4]   2> 84892 INFO  (qtp1728437887-882) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-882]: apply all deletes during flush
   [junit4]   2> 84892 INFO  (qtp1728437887-882) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-882]: now apply all deletes for all segments maxDoc=400
   [junit4]   2> 84892 INFO  (qtp1728437887-882) [    x:collection1] o.a.s.u.LoggingInfoStream [BD][qtp1728437887-882]: applyDeletes: open segment readers took 0 msec
   [junit4]   2> 84892 INFO  (qtp1728437887-882) [    x:collection1] o.a.s.u.LoggingInfoStream [BD][qtp1728437887-882]: applyDeletes: no segments; skipping
   [junit4]   2> 84892 INFO  (qtp1728437887-882) [    x:collection1] o.a.s.u.LoggingInfoStream [BD][qtp1728437887-882]: prune sis=segments_2: _0(7.0.0):C158 _1(7.0.0):C154 _2(7.0.0):C88 minGen=5 packetCount=0
   [junit4]   2> 84892 INFO  (qtp1728437887-882) [    x:collection1] o.a.s.u.LoggingInfoStream [TMP][qtp1728437887-882]: findForcedMerges maxSegmentCount=1 infos=_0(7.0.0):C158 _1(7.0.0):C154 _2(7.0.0):C88 segmentsToMerge={_0(7.0.0):C158=true, _2(7.0.0):C88=true, _1(7.0.0):C154=true}
   [junit4]   2> 84892 INFO  (qtp1728437887-882) [    x:collection1] o.a.s.u.LoggingInfoStream [TMP][qtp1728437887-882]: eligible=[_0(7.0.0):C158, _1(7.0.0):C154, _2(7.0.0):C88]
   [junit4]   2> 84892 INFO  (qtp1728437887-882) [    x:collection1] o.a.s.u.LoggingInfoStream [TMP][qtp1728437887-882]: forceMergeRunning=false
   [junit4]   2> 84892 INFO  (qtp1728437887-882) [    x:collection1] o.a.s.u.LoggingInfoStream [TMP][qtp1728437887-882]: add final merge=_0(7.0.0):C158 _1(7.0.0):C154 _2(7.0.0):C88
   [junit4]   2> 84892 INFO  (qtp1728437887-882) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-882]: add merge to pendingMerges: _0(7.0.0):C158 _1(7.0.0):C154 _2(7.0.0):C88 [total 1 pending]
   [junit4]   2> 84892 INFO  (qtp1728437887-882) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-882]: registerMerge merging= []
   [junit4]   2> 84892 INFO  (qtp1728437887-882) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-882]: registerMerge info=_0(7.0.0):C158
   [junit4]   2> 84892 INFO  (qtp1728437887-882) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-882]: registerMerge info=_1(7.0.0):C154
   [junit4]   2> 84892 INFO  (qtp1728437887-882) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][qtp1728437887-882]: registerMerge info=_2(7.0.0):C88
   [junit4]   2> 84893 INFO  (qtp1728437887-882) [    x:collection1] o.a.s.u.LoggingInfoStream [MS][qtp1728437887-882]: now merge
   [junit4]   2> 84893 INFO  (qtp1728437887-882) [    x:collection1] o.a.s.u.LoggingInfoStream [MS][qtp1728437887-882]:   index: _0(7.0.0):C158 _1(7.0.0):C154 _2(7.0.0):C88
   [junit4]   2> 84893 INFO  (qtp1728437887-882) [    x:collection1] o.a.s.u.LoggingInfoStream [MS][qtp1728437887-882]:   consider merge _0(7.0.0):C158 _1(7.0.0):C154 _2(7.0.0):C88
   [junit4]   2> 84893 INFO  (qtp1728437887-882) [    x:collection1] o.a.s.u.LoggingInfoStream [MS][qtp1728437887-882]:     launch new thread [Lucene Merge Thread #0]
   [junit4]   2> 84893 INFO  (qtp1728437887-882) [    x:collection1] o.a.s.u.LoggingInfoStream [MS][qtp1728437887-882]: updateMergeThreads ioThrottle=true targetMBPerSec=20.0 MB/sec
   [junit4]   2> merge thread Lucene Merge Thread #0 estSize=0.0 MB (written=0.0 MB) runTime=0.0s (stopped=0.0s, paused=0.0s) rate=unlimited
   [junit4]   2>   leave running at Infinity MB/sec
   [junit4]   2> 84893 INFO  (qtp1728437887-882) [    x:collection1] o.a.s.u.LoggingInfoStream [MS][qtp1728437887-882]:   no more merges pending; now return
   [junit4]   2> 84893 INFO  (Lucene Merge Thread #0) [    x:collection1] o.a.s.u.LoggingInfoStream [MS][Lucene Merge Thread #0]:   merge thread: start
   [junit4]   2> 84895 INFO  (Lucene Merge Thread #0) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][Lucene Merge Thread #0]: now apply deletes for 3 merging segments
   [junit4]   2> 84895 INFO  (Lucene Merge Thread #0) [    x:collection1] o.a.s.u.LoggingInfoStream [BD][Lucene Merge Thread #0]: applyDeletes: open segment readers took 0 msec
   [junit4]   2> 84895 INFO  (Lucene Merge Thread #0) [    x:collection1] o.a.s.u.LoggingInfoStream [BD][Lucene Merge Thread #0]: applyDeletes: no segments; skipping
   [junit4]   2> 84895 INFO  (Lucene Merge Thread #0) [    x:collection1] o.a.s.u.LoggingInfoStream [BD][Lucene Merge Thread #0]: prune sis=segments_2: _0(7.0.0):C158 _1(7.0.0):C154 _2(7.0.0):C88 minGen=5 packetCount=0
   [junit4]   2> 84895 INFO  (Lucene Merge Thread #0) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][Lucene Merge Thread #0]: merge seg=_3 _0(7.0.0):C158 _1(7.0.0):C154 _2(7.0.0):C88
   [junit4]   2> 84895 INFO  (Lucene Merge Thread #0) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][Lucene Merge Thread #0]: now merge
   [junit4]   2>   merge=_0(7.0.0):C158 _1(7.0.0):C154 _2(7.0.0):C88
   [junit4]   2>   index=_0(7.0.0):C158 _1(7.0.0):C154 _2(7.0.0):C88
   [junit4]   2> 84895 INFO  (Lucene Merge Thread #0) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][Lucene Merge Thread #0]: merging _0(7.0.0):C158 _1(7.0.0):C154 _2(7.0.0):C88
   [junit4]   2> 84895 INFO  (Lucene Merge Thread #0) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][Lucene Merge Thread #0]: seg=_0(7.0.0):C158 no deletes
   [junit4]   2> 84895 INFO  (Lucene Merge Thread #0) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][Lucene Merge Thread #0]: seg=_1(7.0.0):C154 no deletes
   [junit4]   2> 84895 INFO  (Lucene Merge Thread #0) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][Lucene Merge Thread #0]: seg=_2(7.0.0):C88 no deletes
   [junit4]   2> 84895 INFO  (Lucene Merge Thread #0) [    x:collection1] o.a.s.u.LoggingInfoStream [SM][Lucene Merge Thread #0]: merge store matchedCount=3 vs 3
   [junit4]   2> 84897 INFO  (Lucene Merge Thread #0) [    x:collection1] o.a.s.u.LoggingInfoStream [SM][Lucene Merge Thread #0]: 1 msec to merge stored fields [400 docs]
   [junit4]   2> 84906 INFO  (Lucene Merge Thread #0) [    x:collection1] o.a.s.u.LoggingInfoStream [SM][Lucene Merge Thread #0]: 9 msec to merge postings [400 docs]
   [junit4]   2> 84908 INFO  (Lucene Merge Thread #0) [    x:collection1] o.a.s.u.LoggingInfoStream [SM][Lucene Merge Thread #0]: 1 msec to merge doc values [400 docs]
   [junit4]   2> 84908 INFO  (Lucene Merge Thread #0) [    x:collection1] o.a.s.u.LoggingInfoStream [SM][Lucene Merge Thread #0]: 0 msec to merge points [400 docs]
   [junit4]   2> 84910 INFO  (Lucene Merge Thread #0) [    x:collection1] o.a.s.u.LoggingInfoStream [SM][Lucene Merge Thread #0]: 1 msec to merge norms [400 docs]
   [junit4]   2> 84910 INFO  (Lucene Merge Thread #0) [    x:collection1] o.a.s.u.LoggingInfoStream [SM][Lucene Merge Thread #0]: 0 msec to write field infos [400 docs]
   [junit4]   2> 84910 INFO  (Lucene Merge Thread #0) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][Lucene Merge Thread #0]: merge codec=Lucene62 maxDoc=400; merged segment has no vectors; norms; docValues; prox; freqs; no points; 0.0 sec (0.0 sec stopped, 0.0 sec paused) to merge segment [0.01 MB, 0.71 MB/sec]
   [junit4]   2> 84910 INFO  (Lucene Merge Thread #0) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][Lucene Merge Thread #0]: merged segment size=0.010 MB vs estimate=0.017 MB
   [junit4]   2> 84910 INFO  (Lucene Merge Thread #0) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][Lucene Merge Thread #0]: commitMerge: _0(7.0.0):C158 _1(7.0.0):C154 _2(7.0.0):C88 index=_0(7.0.0):C158 _1(7.0.0):C154 _2(7.0.0):C88
   [junit4]   2> 84910 INFO  (Lucene Merge Thread #0) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][Lucene Merge Thread #0]: commitMergeDeletes _0(7.0.0):C158 _1(7.0.0):C154 _2(7.0.0):C88
   [junit4]   2> 84910 INFO  (Lucene Merge Thread #0) [    x:collection1] o.a.s.u.LoggingInfoStream [IW][Lucene Merge Thread #0]: no new deletes or field updates since merge started
   [junit4]   2> 84910 INFO  (Lucene Merge Thread #0) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][Lucene Merge Thread #0]: now checkpoint "_3(7.0.0):C400" [1 segments ; isCommit = false]
   [junit4]   2> 84910 INFO  (Lucene Merge Thread #0) [    x:collection1] o.a.s.u.LoggingInfoStream [IFD][Lucene Merge Thread #0]: delete []
   [junit4]   2> 84910 INFO  (L

[...truncated too long message...]

/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001/collection1/data;done=false>>]
   [junit4]   2> 85110 INFO  (coreCloseExecutor-158-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory, CoreContainer#isShutdown=true
   [junit4]   2> 85110 INFO  (coreCloseExecutor-158-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.embedded.LargeVolumeBinaryJettyTest_2648413B1B80B622-001/tempDir-001/collection1/data
   [junit4]   2> 85117 INFO  (SUITE-LargeVolumeBinaryJettyTest-seed#[2648413B1B80B622]-worker) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@4874c9c4{/solr,null,UNAVAILABLE}
   [junit4]   2> 85141 INFO  (SUITE-LargeVolumeBinaryJettyTest-seed#[2648413B1B80B622]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> ObjectTracker found 1 object(s) that were not released!!! [TransactionLog]
   [junit4]   2> org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
   [junit4]   2> 	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
   [junit4]   2> 	at org.apache.solr.update.TransactionLog.<init>(TransactionLog.java:188)
   [junit4]   2> 	at org.apache.solr.update.UpdateLog.newTransactionLog(UpdateLog.java:344)
   [junit4]   2> 	at org.apache.solr.update.UpdateLog.ensureLog(UpdateLog.java:857)
   [junit4]   2> 	at org.apache.solr.update.UpdateLog.add(UpdateLog.java:428)
   [junit4]   2> 	at org.apache.solr.update.UpdateLog.add(UpdateLog.java:415)
   [junit4]   2> 	at org.apache.solr.update.DirectUpdateHandler2.doNormalUpdate(DirectUpdateHandler2.java:302)
   [junit4]   2> 	at org.apache.solr.update.DirectUpdateHandler2.addDoc0(DirectUpdateHandler2.java:214)
   [junit4]   2> 	at org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:169)
   [junit4]   2> 	at org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:67)
   [junit4]   2> 	at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48)
   [junit4]   2> 	at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd(DistributedUpdateProcessor.java:939)
   [junit4]   2> 	at org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:1094)
   [junit4]   2> 	at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:720)
   [junit4]   2> 	at org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:103)
   [junit4]   2> 	at org.apache.solr.handler.loader.JavabinLoader$1.update(JavabinLoader.java:97)
   [junit4]   2> 	at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readOuterMostDocIterator(JavaBinUpdateRequestCodec.java:179)
   [junit4]   2> 	at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readIterator(JavaBinUpdateRequestCodec.java:135)
   [junit4]   2> 	at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:274)
   [junit4]   2> 	at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readNamedList(JavaBinUpdateRequestCodec.java:121)
   [junit4]   2> 	at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:239)
   [junit4]   2> 	at org.apache.solr.common.util.JavaBinCodec.unmarshal(JavaBinCodec.java:157)
   [junit4]   2> 	at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.unmarshal(JavaBinUpdateRequestCodec.java:186)
   [junit4]   2> 	at org.apache.solr.handler.loader.JavabinLoader.parseAndLoadDocs(JavabinLoader.java:107)
   [junit4]   2> 	at org.apache.solr.handler.loader.JavabinLoader.load(JavabinLoader.java:54)
   [junit4]   2> 	at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:97)
   [junit4]   2> 	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:68)
   [junit4]   2> 	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:154)
   [junit4]   2> 	at org.apache.solr.core.SolrCore.execute(SolrCore.java:2035)
   [junit4]   2> 	at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:653)
   [junit4]   2> 	at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:460)
   [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)
   [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
   [junit4]   2> 	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:111)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
   [junit4]   2> 	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
   [junit4]   2> 	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
   [junit4]   2> 	at org.eclipse.jetty.server.Server.handle(Server.java:518)
   [junit4]   2> 	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
   [junit4]   2> 	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
   [junit4]   2> 	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
   [junit4]   2> 	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
   [junit4]   2> 	at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene62): {}, docValues:{}, maxPointsInLeafNode=1853, maxMBSortInHeap=6.253214419428486, sim=ClassicSimilarity, locale=es-CR, timezone=America/Eirunepe
   [junit4]   2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_92 (64-bit)/cpus=3,threads=1,free=197566720,total=331350016
   [junit4]   2> NOTE: All tests run in this JVM: [SolrExceptionTest, StreamExpressionToExplanationTest, StreamExpressionParserTest, OperationsTest, BasicHttpSolrClientTest, SolrQueryTest, ContentStreamTest, TestNamedListCodec, FacetFieldTest, ClientUtilsTest, TestJsonRecordReader, ConcurrentUpdateSolrClientBuilderTest, JdbcTest, CommonParamsTest, DocumentAnalysisResponseTest, GraphTest, SolrPingTest, TestFastInputStream, SolrZkClientTest, SolrExampleStreamingTest, SolrExampleStreamingBinaryTest, SolrExampleBinaryTest, SolrExampleXMLTest, SolrExampleJettyTest, LargeVolumeBinaryJettyTest]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=LargeVolumeBinaryJettyTest -Dtests.seed=2648413B1B80B622 -Dtests.slow=true -Dtests.locale=es-CR -Dtests.timezone=America/Eirunepe -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.00s J1 | LargeVolumeBinaryJettyTest (suite) <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: ObjectTracker found 1 object(s) that were not released!!! [TransactionLog]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([2648413B1B80B622]:0)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:256)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4] Completed [47/83 (1!)] on J1 in 1.96s, 1 test, 1 failure <<< FAILURES!

[...truncated 118 lines...]
BUILD FAILED
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/build.xml:740: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/build.xml:684: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/build.xml:59: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build.xml:246: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/common-build.xml:530: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/lucene/common-build.xml:1427: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/lucene/common-build.xml:984: There were test failures: 83 suites, 645 tests, 1 suite-level error, 2 ignored [seed: 2648413B1B80B622]

Total time: 75 minutes 7 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 694 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/694/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.cloud.overseer.ZkStateWriterTest.testExternalModificationToStateFormat2

Error Message:


Stack Trace:
java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([D3B9487A7DF2BFB8:A2645CAE1C594338]:0)
	at org.junit.Assert.fail(Assert.java:92)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertTrue(Assert.java:54)
	at org.apache.solr.cloud.overseer.ZkStateWriterTest.testExternalModificationToStateFormat2(ZkStateWriterTest.java:322)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10583 lines...]
   [junit4] Suite: org.apache.solr.cloud.overseer.ZkStateWriterTest
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.overseer.ZkStateWriterTest_D3B9487A7DF2BFB8-001/init-core-data-001
   [junit4]   2> 230324 INFO  (SUITE-ZkStateWriterTest-seed#[D3B9487A7DF2BFB8]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 230328 INFO  (TEST-ZkStateWriterTest.testSingleLegacyCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testSingleLegacyCollection
   [junit4]   2> 230329 INFO  (TEST-ZkStateWriterTest.testSingleLegacyCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 230329 INFO  (Thread-2007) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 230329 INFO  (Thread-2007) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 230429 INFO  (TEST-ZkStateWriterTest.testSingleLegacyCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.ZkTestServer start zk server on port:41876
   [junit4]   2> 230429 INFO  (TEST-ZkStateWriterTest.testSingleLegacyCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 230430 INFO  (TEST-ZkStateWriterTest.testSingleLegacyCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 230433 INFO  (zkCallback-257-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@afe159e name:ZooKeeperConnection Watcher:127.0.0.1:41876 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 230433 INFO  (TEST-ZkStateWriterTest.testSingleLegacyCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 230433 INFO  (TEST-ZkStateWriterTest.testSingleLegacyCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 230435 INFO  (TEST-ZkStateWriterTest.testSingleLegacyCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 230435 INFO  (TEST-ZkStateWriterTest.testSingleLegacyCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 230437 INFO  (zkCallback-258-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5ef5b80c name:ZooKeeperConnection Watcher:127.0.0.1:41876 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 230437 INFO  (TEST-ZkStateWriterTest.testSingleLegacyCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 230438 INFO  (TEST-ZkStateWriterTest.testSingleLegacyCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 230438 INFO  (TEST-ZkStateWriterTest.testSingleLegacyCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 230443 INFO  (TEST-ZkStateWriterTest.testSingleLegacyCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 230444 INFO  (TEST-ZkStateWriterTest.testSingleLegacyCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 230445 INFO  (zkCallback-259-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@61299f7 name:ZooKeeperConnection Watcher:127.0.0.1:41876/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 230446 INFO  (TEST-ZkStateWriterTest.testSingleLegacyCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 230446 INFO  (TEST-ZkStateWriterTest.testSingleLegacyCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 230447 INFO  (TEST-ZkStateWriterTest.testSingleLegacyCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 230449 INFO  (TEST-ZkStateWriterTest.testSingleLegacyCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections
   [junit4]   2> 230451 INFO  (TEST-ZkStateWriterTest.testSingleLegacyCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 230453 INFO  (TEST-ZkStateWriterTest.testSingleLegacyCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 230456 INFO  (TEST-ZkStateWriterTest.testSingleLegacyCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 230457 INFO  (TEST-ZkStateWriterTest.testSingleLegacyCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 230458 INFO  (TEST-ZkStateWriterTest.testSingleLegacyCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 230459 INFO  (TEST-ZkStateWriterTest.testSingleLegacyCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 230461 INFO  (TEST-ZkStateWriterTest.testSingleLegacyCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/c1
   [junit4]   2> 230463 INFO  (zkCallback-259-thread-1) [    ] o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections], has occurred - updating...
   [junit4]   2> 230466 INFO  (zkCallback-259-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 230469 INFO  (TEST-ZkStateWriterTest.testSingleLegacyCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:41876 41876
   [junit4]   2> 230503 INFO  (Thread-2007) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:41876 41876
   [junit4]   2> 230504 INFO  (TEST-ZkStateWriterTest.testSingleLegacyCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testSingleLegacyCollection
   [junit4]   2> 230510 INFO  (TEST-ZkStateWriterTest.testZkStateWriterBatching-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testZkStateWriterBatching
   [junit4]   2> 230511 INFO  (TEST-ZkStateWriterTest.testZkStateWriterBatching-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 230511 INFO  (Thread-2008) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 230511 INFO  (Thread-2008) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 230611 INFO  (TEST-ZkStateWriterTest.testZkStateWriterBatching-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.ZkTestServer start zk server on port:62072
   [junit4]   2> 230611 INFO  (TEST-ZkStateWriterTest.testZkStateWriterBatching-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 230612 INFO  (TEST-ZkStateWriterTest.testZkStateWriterBatching-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 230615 INFO  (zkCallback-261-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@46ba015 name:ZooKeeperConnection Watcher:127.0.0.1:62072 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 230615 INFO  (TEST-ZkStateWriterTest.testZkStateWriterBatching-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 230616 INFO  (TEST-ZkStateWriterTest.testZkStateWriterBatching-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 230617 INFO  (TEST-ZkStateWriterTest.testZkStateWriterBatching-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 230618 INFO  (TEST-ZkStateWriterTest.testZkStateWriterBatching-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 230619 INFO  (zkCallback-262-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@42091870 name:ZooKeeperConnection Watcher:127.0.0.1:62072 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 230619 INFO  (TEST-ZkStateWriterTest.testZkStateWriterBatching-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 230619 INFO  (TEST-ZkStateWriterTest.testZkStateWriterBatching-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 230620 INFO  (TEST-ZkStateWriterTest.testZkStateWriterBatching-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 230624 INFO  (TEST-ZkStateWriterTest.testZkStateWriterBatching-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 230624 INFO  (TEST-ZkStateWriterTest.testZkStateWriterBatching-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 230626 INFO  (zkCallback-263-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@21acce16 name:ZooKeeperConnection Watcher:127.0.0.1:62072/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 230626 INFO  (TEST-ZkStateWriterTest.testZkStateWriterBatching-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 230627 INFO  (TEST-ZkStateWriterTest.testZkStateWriterBatching-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 230628 INFO  (TEST-ZkStateWriterTest.testZkStateWriterBatching-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 230631 INFO  (TEST-ZkStateWriterTest.testZkStateWriterBatching-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections
   [junit4]   2> 230633 INFO  (TEST-ZkStateWriterTest.testZkStateWriterBatching-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 230635 INFO  (TEST-ZkStateWriterTest.testZkStateWriterBatching-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 230638 INFO  (TEST-ZkStateWriterTest.testZkStateWriterBatching-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 230640 INFO  (TEST-ZkStateWriterTest.testZkStateWriterBatching-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 230641 INFO  (TEST-ZkStateWriterTest.testZkStateWriterBatching-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 230642 INFO  (TEST-ZkStateWriterTest.testZkStateWriterBatching-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 230645 INFO  (TEST-ZkStateWriterTest.testZkStateWriterBatching-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/c1
   [junit4]   2> 230647 INFO  (zkCallback-263-thread-1) [    ] o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections], has occurred - updating...
   [junit4]   2> 230647 INFO  (TEST-ZkStateWriterTest.testZkStateWriterBatching-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/c2
   [junit4]   2> 230650 INFO  (zkCallback-263-thread-1) [    ] o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections], has occurred - updating...
   [junit4]   2> 230652 INFO  (TEST-ZkStateWriterTest.testZkStateWriterBatching-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/c1/state.json
   [junit4]   2> 230655 INFO  (TEST-ZkStateWriterTest.testZkStateWriterBatching-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:62072 62072
   [junit4]   2> 230755 INFO  (Thread-2008) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:62072 62072
   [junit4]   2> 230757 INFO  (TEST-ZkStateWriterTest.testZkStateWriterBatching-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testZkStateWriterBatching
   [junit4]   2> 230762 INFO  (TEST-ZkStateWriterTest.testSingleExternalCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testSingleExternalCollection
   [junit4]   2> 230763 INFO  (TEST-ZkStateWriterTest.testSingleExternalCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 230763 INFO  (Thread-2009) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 230763 INFO  (Thread-2009) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 230863 INFO  (TEST-ZkStateWriterTest.testSingleExternalCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.ZkTestServer start zk server on port:57167
   [junit4]   2> 230863 INFO  (TEST-ZkStateWriterTest.testSingleExternalCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 230864 INFO  (TEST-ZkStateWriterTest.testSingleExternalCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 230871 INFO  (zkCallback-265-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3cf15e44 name:ZooKeeperConnection Watcher:127.0.0.1:57167 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 230872 INFO  (TEST-ZkStateWriterTest.testSingleExternalCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 230872 INFO  (TEST-ZkStateWriterTest.testSingleExternalCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 230874 INFO  (TEST-ZkStateWriterTest.testSingleExternalCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 230875 INFO  (TEST-ZkStateWriterTest.testSingleExternalCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 230877 INFO  (zkCallback-266-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1d781b68 name:ZooKeeperConnection Watcher:127.0.0.1:57167 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 230877 INFO  (TEST-ZkStateWriterTest.testSingleExternalCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 230877 INFO  (TEST-ZkStateWriterTest.testSingleExternalCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 230877 INFO  (TEST-ZkStateWriterTest.testSingleExternalCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 230885 INFO  (TEST-ZkStateWriterTest.testSingleExternalCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 230885 INFO  (TEST-ZkStateWriterTest.testSingleExternalCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 230887 INFO  (zkCallback-267-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@66f1b524 name:ZooKeeperConnection Watcher:127.0.0.1:57167/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 230887 INFO  (TEST-ZkStateWriterTest.testSingleExternalCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 230887 INFO  (TEST-ZkStateWriterTest.testSingleExternalCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 230888 INFO  (TEST-ZkStateWriterTest.testSingleExternalCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 230891 INFO  (TEST-ZkStateWriterTest.testSingleExternalCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections
   [junit4]   2> 230893 INFO  (TEST-ZkStateWriterTest.testSingleExternalCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 230895 INFO  (TEST-ZkStateWriterTest.testSingleExternalCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 230897 INFO  (TEST-ZkStateWriterTest.testSingleExternalCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 230899 INFO  (TEST-ZkStateWriterTest.testSingleExternalCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 230900 INFO  (TEST-ZkStateWriterTest.testSingleExternalCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 230901 INFO  (TEST-ZkStateWriterTest.testSingleExternalCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 230903 INFO  (TEST-ZkStateWriterTest.testSingleExternalCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/c1
   [junit4]   2> 230906 INFO  (zkCallback-267-thread-1) [    ] o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections], has occurred - updating...
   [junit4]   2> 230908 INFO  (TEST-ZkStateWriterTest.testSingleExternalCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/c1/state.json
   [junit4]   2> 230913 INFO  (TEST-ZkStateWriterTest.testSingleExternalCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:57167 57167
   [junit4]   2> 230927 INFO  (Thread-2009) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:57167 57167
   [junit4]   2> 230929 INFO  (TEST-ZkStateWriterTest.testSingleExternalCollection-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testSingleExternalCollection
   [junit4]   2> 230935 INFO  (TEST-ZkStateWriterTest.testExternalModificationToSharedClusterState-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testExternalModificationToSharedClusterState
   [junit4]   2> 230936 INFO  (TEST-ZkStateWriterTest.testExternalModificationToSharedClusterState-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 230939 INFO  (Thread-2010) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 230939 INFO  (Thread-2010) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 231039 INFO  (TEST-ZkStateWriterTest.testExternalModificationToSharedClusterState-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.ZkTestServer start zk server on port:35856
   [junit4]   2> 231039 INFO  (TEST-ZkStateWriterTest.testExternalModificationToSharedClusterState-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 231040 INFO  (TEST-ZkStateWriterTest.testExternalModificationToSharedClusterState-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 231043 INFO  (zkCallback-269-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1dbcce64 name:ZooKeeperConnection Watcher:127.0.0.1:35856 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 231044 INFO  (TEST-ZkStateWriterTest.testExternalModificationToSharedClusterState-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 231044 INFO  (TEST-ZkStateWriterTest.testExternalModificationToSharedClusterState-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 231047 INFO  (TEST-ZkStateWriterTest.testExternalModificationToSharedClusterState-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 231048 INFO  (TEST-ZkStateWriterTest.testExternalModificationToSharedClusterState-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 231052 INFO  (zkCallback-270-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4b24f4b2 name:ZooKeeperConnection Watcher:127.0.0.1:35856 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 231052 INFO  (TEST-ZkStateWriterTest.testExternalModificationToSharedClusterState-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 231053 INFO  (TEST-ZkStateWriterTest.testExternalModificationToSharedClusterState-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 231053 INFO  (TEST-ZkStateWriterTest.testExternalModificationToSharedClusterState-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 231062 INFO  (TEST-ZkStateWriterTest.testExternalModificationToSharedClusterState-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 231062 INFO  (TEST-ZkStateWriterTest.testExternalModificationToSharedClusterState-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 231064 INFO  (zkCallback-271-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@66edaf72 name:ZooKeeperConnection Watcher:127.0.0.1:35856/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 231065 INFO  (TEST-ZkStateWriterTest.testExternalModificationToSharedClusterState-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 231065 INFO  (TEST-ZkStateWriterTest.testExternalModificationToSharedClusterState-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 231070 INFO  (TEST-ZkStateWriterTest.testExternalModificationToSharedClusterState-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 231084 INFO  (TEST-ZkStateWriterTest.testExternalModificationToSharedClusterState-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections
   [junit4]   2> 231088 INFO  (TEST-ZkStateWriterTest.testExternalModificationToSharedClusterState-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 231091 INFO  (TEST-ZkStateWriterTest.testExternalModificationToSharedClusterState-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 231094 INFO  (TEST-ZkStateWriterTest.testExternalModificationToSharedClusterState-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 231098 INFO  (TEST-ZkStateWriterTest.testExternalModificationToSharedClusterState-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 231100 INFO  (TEST-ZkStateWriterTest.testExternalModificationToSharedClusterState-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 231107 INFO  (TEST-ZkStateWriterTest.testExternalModificationToSharedClusterState-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 231111 INFO  (TEST-ZkStateWriterTest.testExternalModificationToSharedClusterState-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/c1
   [junit4]   2> 231117 INFO  (zkCallback-271-thread-1) [    ] o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections], has occurred - updating...
   [junit4]   2> 231119 INFO  (TEST-ZkStateWriterTest.testExternalModificationToSharedClusterState-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/c2
   [junit4]   2> 231122 INFO  (zkCallback-271-thread-1) [    ] o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections], has occurred - updating...
   [junit4]   2> 231125 INFO  (zkCallback-271-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 231133 INFO  (zkCallback-271-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 231138 INFO  (TEST-ZkStateWriterTest.testExternalModificationToSharedClusterState-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:35856 35856
   [junit4]   2> 231254 INFO  (Thread-2010) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:35856 35856
   [junit4]   2> 231259 INFO  (TEST-ZkStateWriterTest.testExternalModificationToSharedClusterState-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testExternalModificationToSharedClusterState
   [junit4]   2> 231267 INFO  (TEST-ZkStateWriterTest.testExternalModificationToStateFormat2-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testExternalModificationToStateFormat2
   [junit4]   2> 231268 INFO  (TEST-ZkStateWriterTest.testExternalModificationToStateFormat2-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 231268 INFO  (Thread-2011) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 231268 INFO  (Thread-2011) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 231369 INFO  (TEST-ZkStateWriterTest.testExternalModificationToStateFormat2-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.ZkTestServer start zk server on port:47804
   [junit4]   2> 231369 INFO  (TEST-ZkStateWriterTest.testExternalModificationToStateFormat2-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 231370 INFO  (TEST-ZkStateWriterTest.testExternalModificationToStateFormat2-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 231380 INFO  (zkCallback-273-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@73f908e7 name:ZooKeeperConnection Watcher:127.0.0.1:47804 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 231381 INFO  (TEST-ZkStateWriterTest.testExternalModificationToStateFormat2-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 231381 INFO  (TEST-ZkStateWriterTest.testExternalModificationToStateFormat2-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 231384 INFO  (TEST-ZkStateWriterTest.testExternalModificationToStateFormat2-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 231385 INFO  (TEST-ZkStateWriterTest.testExternalModificationToStateFormat2-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 231387 INFO  (zkCallback-274-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@42351845 name:ZooKeeperConnection Watcher:127.0.0.1:47804 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 231387 INFO  (TEST-ZkStateWriterTest.testExternalModificationToStateFormat2-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 231387 INFO  (TEST-ZkStateWriterTest.testExternalModificationToStateFormat2-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 231387 INFO  (TEST-ZkStateWriterTest.testExternalModificationToStateFormat2-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 231407 INFO  (TEST-ZkStateWriterTest.testExternalModificationToStateFormat2-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 231408 INFO  (TEST-ZkStateWriterTest.testExternalModificationToStateFormat2-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 231412 INFO  (zkCallback-275-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@21e5a556 name:ZooKeeperConnection Watcher:127.0.0.1:47804/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 231412 INFO  (TEST-ZkStateWriterTest.testExternalModificationToStateFormat2-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 231413 INFO  (TEST-ZkStateWriterTest.testExternalModificationToStateFormat2-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 231415 INFO  (TEST-ZkStateWriterTest.testExternalModificationToStateFormat2-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 231421 INFO  (TEST-ZkStateWriterTest.testExternalModificationToStateFormat2-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections
   [junit4]   2> 231426 INFO  (TEST-ZkStateWriterTest.testExternalModificationToStateFormat2-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 231429 INFO  (TEST-ZkStateWriterTest.testExternalModificationToStateFormat2-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 231432 INFO  (TEST-ZkStateWriterTest.testExternalModificationToStateFormat2-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 231435 INFO  (TEST-ZkStateWriterTest.testExternalModificationToStateFormat2-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 231436 INFO  (TEST-ZkStateWriterTest.testExternalModificationToStateFormat2-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 231438 INFO  (TEST-ZkStateWriterTest.testExternalModificationToStateFormat2-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 231441 INFO  (TEST-ZkStateWriterTest.testExternalModificationToStateFormat2-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/c1
   [junit4]   2> 231444 INFO  (zkCallback-275-thread-1) [    ] o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections], has occurred - updating...
   [junit4]   2> 231445 INFO  (TEST-ZkStateWriterTest.testExternalModificationToStateFormat2-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/c2
   [junit4]   2> 231448 INFO  (zkCallback-275-thread-1) [    ] o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections], has occurred - updating...
   [junit4]   2> 231450 INFO  (TEST-ZkStateWriterTest.testExternalModificationToStateFormat2-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/c2/state.json
   [junit4]   2> 231463 INFO  (TEST-ZkStateWriterTest.testExternalModificationToStateFormat2-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:47804 47804
   [junit4]   2> 231621 INFO  (Thread-2011) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:47804 47804
   [junit4]   2> 231624 INFO  (TEST-ZkStateWriterTest.testExternalModificationToStateFormat2-seed#[D3B9487A7DF2BFB8]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testExternalModificationToStateFormat2
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=ZkStateWriterTest -Dtests.method=testExternalModificationToStateFormat2 -Dtests.seed=D3B9487A7DF2BFB8 -Dtests.slow=true -Dtests.locale=fi -Dtests.timezone=America/Monterrey -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE 0.37s J1 | ZkStateWriterTest.testExternalModificationToStateFormat2 <<<
   [junit4]    > Throwable #1: java.lang.AssertionError
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([D3B9487A7DF2BFB8:A2645CAE1C594338]:0)
   [junit4]    > 	at org.apache.solr.cloud.overseer.ZkStateWriterTest.testExternalModificationToStateFormat2(ZkStateWriterTest.java:322)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 231633 INFO  (SUITE-ZkStateWriterTest-seed#[D3B9487A7DF2BFB8]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.overseer.ZkStateWriterTest_D3B9487A7DF2BFB8-001
   [junit4]   2> NOTE: test params are: codec=FastDecompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=5, maxDocsPerChunk=737, blockSize=4), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=5, blockSize=4)), sim=ClassicSimilarity, locale=fi, timezone=America/Monterrey
   [junit4]   2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_92 (64-bit)/cpus=3,threads=1,free=157016024,total=417030144
   [junit4]   2> NOTE: All tests run in this JVM: [ConvertedLegacyTest, NumericFieldsTest, TestDistributedGrouping, TestReversedWildcardFilterFactory, CloudMLTQParserTest, RequestHandlersTest, TestCloudDeleteByQuery, TestTrieFacet, TestSolrDeletionPolicy2, LeaderFailoverAfterPartitionTest, CheckHdfsIndexTest, PluginInfoTest, TestInitQParser, ResponseHeaderTest, LeaderInitiatedRecoveryOnShardRestartTest, ChaosMonkeySafeLeaderTest, TestSerializedLuceneMatchVersion, ZkStateWriterTest]
   [junit4] Completed [35/618 (1!)] on J1 in 1.34s, 5 tests, 1 failure <<< FAILURES!

[...truncated 1892 lines...]
BUILD FAILED
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/build.xml:740: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/build.xml:684: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/build.xml:59: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build.xml:233: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/common-build.xml:530: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/lucene/common-build.xml:1427: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/lucene/common-build.xml:984: There were test failures: 618 suites (10 ignored), 2588 tests, 1 failure, 85 ignored (70 assumptions) [seed: D3B9487A7DF2BFB8]

Total time: 79 minutes 23 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 693 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/693/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseG1GC

3 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.DistributedVersionInfoTest

Error Message:
java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:37674 within 30000 ms

Stack Trace:
org.apache.solr.common.SolrException: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:37674 within 30000 ms
	at __randomizedtesting.SeedInfo.seed([1A7ADE7FF2D37404]:0)
	at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:182)
	at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:116)
	at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:111)
	at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:98)
	at org.apache.solr.cloud.MiniSolrCloudCluster.waitForAllNodes(MiniSolrCloudCluster.java:232)
	at org.apache.solr.cloud.MiniSolrCloudCluster.<init>(MiniSolrCloudCluster.java:226)
	at org.apache.solr.cloud.MiniSolrCloudCluster.<init>(MiniSolrCloudCluster.java:170)
	at org.apache.solr.cloud.SolrCloudTestCase$Builder.configure(SolrCloudTestCase.java:134)
	at org.apache.solr.cloud.DistributedVersionInfoTest.setupCluster(DistributedVersionInfoTest.java:72)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:811)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:37674 within 30000 ms
	at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:228)
	at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:174)
	... 32 more


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.DistributedVersionInfoTest

Error Message:
35 threads leaked from SUITE scope at org.apache.solr.cloud.DistributedVersionInfoTest:     1) Thread[id=7493, name=jetty-launcher-831-thread-1, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=7514, name=qtp22243454-7514, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:546)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:609)         at java.lang.Thread.run(Thread.java:745)    3) Thread[id=7505, name=qtp593071769-7505, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:546)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:609)         at java.lang.Thread.run(Thread.java:745)    4) Thread[id=7518, name=qtp22243454-7518, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:546)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:609)         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=7513, name=qtp1302711442-7513, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:546)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:609)         at java.lang.Thread.run(Thread.java:745)    6) Thread[id=7509, name=qtp1302711442-7509, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:546)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:609)         at java.lang.Thread.run(Thread.java:745)    7) Thread[id=7495, name=jetty-launcher-831-thread-3, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    8) Thread[id=7522, name=org.eclipse.jetty.server.session.HashSessionManager@1a8e8342Timer, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    9) Thread[id=7521, name=org.eclipse.jetty.server.session.HashSessionManager@15643d74Timer, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   10) Thread[id=7489, name=ProcessThread(sid:0 cport:-1):, state=WAITING, group=TGRP-DistributedVersionInfoTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:120)   11) Thread[id=7508, name=qtp22243454-7508, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:546)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:609)         at java.lang.Thread.run(Thread.java:745)   12) Thread[id=7502, name=qtp22243454-7502, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:546)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:609)         at java.lang.Thread.run(Thread.java:745)   13) Thread[id=7520, name=qtp1302711442-7520, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:546)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:609)         at java.lang.Thread.run(Thread.java:745)   14) Thread[id=7504, name=qtp593071769-7504, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:546)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:609)         at java.lang.Thread.run(Thread.java:745)   15) Thread[id=7486, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-DistributedVersionInfoTest]         at sun.nio.ch.DevPollArrayWrapper.poll0(Native Method)         at sun.nio.ch.DevPollArrayWrapper.poll(DevPollArrayWrapper.java:223)         at sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:98)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:178)         at java.lang.Thread.run(Thread.java:745)   16) Thread[id=7499, name=qtp1302711442-7499, state=RUNNABLE, group=TGRP-DistributedVersionInfoTest]         at sun.nio.ch.DevPollArrayWrapper.poll0(Native Method)         at sun.nio.ch.DevPollArrayWrapper.poll(DevPollArrayWrapper.java:223)         at sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:98)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:233)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:181)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:171)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:102)         at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:137)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)         at java.lang.Thread.run(Thread.java:745)   17) Thread[id=7519, name=qtp1302711442-7519, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:546)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:609)         at java.lang.Thread.run(Thread.java:745)   18) Thread[id=7510, name=qtp1302711442-7510, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:546)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:609)         at java.lang.Thread.run(Thread.java:745)   19) Thread[id=7512, name=qtp1302711442-7512, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:546)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:609)         at java.lang.Thread.run(Thread.java:745)   20) Thread[id=7529, name=Scheduler-261391908, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   21) Thread[id=7507, name=qtp1302711442-7507-acceptor-0@65eebb6c-ServerConnector@18886709{HTTP/1.1,[http/1.1]}{127.0.0.1:34146}, state=RUNNABLE, group=TGRP-DistributedVersionInfoTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)         at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:382)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:593)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)         at java.lang.Thread.run(Thread.java:745)   22) Thread[id=7500, name=qtp593071769-7500, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:546)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:609)         at java.lang.Thread.run(Thread.java:745)   23) Thread[id=7503, name=qtp593071769-7503, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:546)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:609)         at java.lang.Thread.run(Thread.java:745)   24) Thread[id=7497, name=qtp593071769-7497-acceptor-0@5431fb37-ServerConnector@778746e3{HTTP/1.1,[http/1.1]}{127.0.0.1:43197}, state=RUNNABLE, group=TGRP-DistributedVersionInfoTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)         at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:382)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:593)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)         at java.lang.Thread.run(Thread.java:745)   25) Thread[id=7511, name=org.eclipse.jetty.server.session.HashSessionManager@5c46d9b3Timer, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   26) Thread[id=7515, name=qtp22243454-7515, state=RUNNABLE, group=TGRP-DistributedVersionInfoTest]         at sun.nio.ch.DevPollArrayWrapper.poll0(Native Method)         at sun.nio.ch.DevPollArrayWrapper.poll(DevPollArrayWrapper.java:223)         at sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:98)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:233)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:181)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:171)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)         at java.lang.Thread.run(Thread.java:745)   27) Thread[id=7485, name=Thread-3871, state=WAITING, group=TGRP-DistributedVersionInfoTest]         at java.lang.Object.wait(Native Method)         at java.lang.Thread.join(Thread.java:1245)         at java.lang.Thread.join(Thread.java:1319)         at org.apache.zookeeper.server.NIOServerCnxnFactory.join(NIOServerCnxnFactory.java:302)         at org.apache.solr.cloud.ZkTestServer$ZKServerMain.runFromConfig(ZkTestServer.java:310)         at org.apache.solr.cloud.ZkTestServer$2.run(ZkTestServer.java:491)   28) Thread[id=7501, name=qtp593071769-7501, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:546)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:609)         at java.lang.Thread.run(Thread.java:745)   29) Thread[id=7517, name=qtp22243454-7517, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:546)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:609)         at java.lang.Thread.run(Thread.java:745)   30) Thread[id=7498, name=qtp593071769-7498, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:546)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:609)         at java.lang.Thread.run(Thread.java:745)   31) Thread[id=7494, name=jetty-launcher-831-thread-2, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   32) Thread[id=7487, name=SessionTracker, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]         at java.lang.Object.wait(Native Method)         at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146)   33) Thread[id=7516, name=qtp22243454-7516, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:546)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:609)         at java.lang.Thread.run(Thread.java:745)   34) Thread[id=7496, name=qtp593071769-7496, state=RUNNABLE, group=TGRP-DistributedVersionInfoTest]         at sun.nio.ch.DevPollArrayWrapper.poll0(Native Method)         at sun.nio.ch.DevPollArrayWrapper.poll(DevPollArrayWrapper.java:223)         at sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:98)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:233)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:181)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:171)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:102)         at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:137)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)         at java.lang.Thread.run(Thread.java:745)   35) Thread[id=7506, name=qtp22243454-7506-acceptor-0@116cf429-ServerConnector@12f8ebf0{HTTP/1.1,[http/1.1]}{127.0.0.1:37674}, state=RUNNABLE, group=TGRP-DistributedVersionInfoTest]         at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)         at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)         at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:382)         at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:593)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 35 threads leaked from SUITE scope at org.apache.solr.cloud.DistributedVersionInfoTest: 
   1) Thread[id=7493, name=jetty-launcher-831-thread-1, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=7514, name=qtp22243454-7514, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:546)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:609)
        at java.lang.Thread.run(Thread.java:745)
   3) Thread[id=7505, name=qtp593071769-7505, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:546)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:609)
        at java.lang.Thread.run(Thread.java:745)
   4) Thread[id=7518, name=qtp22243454-7518, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:546)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:609)
        at java.lang.Thread.run(Thread.java:745)
   5) Thread[id=7513, name=qtp1302711442-7513, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:546)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:609)
        at java.lang.Thread.run(Thread.java:745)
   6) Thread[id=7509, name=qtp1302711442-7509, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:546)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:609)
        at java.lang.Thread.run(Thread.java:745)
   7) Thread[id=7495, name=jetty-launcher-831-thread-3, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   8) Thread[id=7522, name=org.eclipse.jetty.server.session.HashSessionManager@1a8e8342Timer, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   9) Thread[id=7521, name=org.eclipse.jetty.server.session.HashSessionManager@15643d74Timer, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
  10) Thread[id=7489, name=ProcessThread(sid:0 cport:-1):, state=WAITING, group=TGRP-DistributedVersionInfoTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:120)
  11) Thread[id=7508, name=qtp22243454-7508, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:546)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:609)
        at java.lang.Thread.run(Thread.java:745)
  12) Thread[id=7502, name=qtp22243454-7502, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:546)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:609)
        at java.lang.Thread.run(Thread.java:745)
  13) Thread[id=7520, name=qtp1302711442-7520, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:546)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:609)
        at java.lang.Thread.run(Thread.java:745)
  14) Thread[id=7504, name=qtp593071769-7504, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:546)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:609)
        at java.lang.Thread.run(Thread.java:745)
  15) Thread[id=7486, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-DistributedVersionInfoTest]
        at sun.nio.ch.DevPollArrayWrapper.poll0(Native Method)
        at sun.nio.ch.DevPollArrayWrapper.poll(DevPollArrayWrapper.java:223)
        at sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:98)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:178)
        at java.lang.Thread.run(Thread.java:745)
  16) Thread[id=7499, name=qtp1302711442-7499, state=RUNNABLE, group=TGRP-DistributedVersionInfoTest]
        at sun.nio.ch.DevPollArrayWrapper.poll0(Native Method)
        at sun.nio.ch.DevPollArrayWrapper.poll(DevPollArrayWrapper.java:223)
        at sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:98)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:233)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:181)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:171)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:102)
        at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:137)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
        at java.lang.Thread.run(Thread.java:745)
  17) Thread[id=7519, name=qtp1302711442-7519, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:546)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:609)
        at java.lang.Thread.run(Thread.java:745)
  18) Thread[id=7510, name=qtp1302711442-7510, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:546)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:609)
        at java.lang.Thread.run(Thread.java:745)
  19) Thread[id=7512, name=qtp1302711442-7512, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:546)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:609)
        at java.lang.Thread.run(Thread.java:745)
  20) Thread[id=7529, name=Scheduler-261391908, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
  21) Thread[id=7507, name=qtp1302711442-7507-acceptor-0@65eebb6c-ServerConnector@18886709{HTTP/1.1,[http/1.1]}{127.0.0.1:34146}, state=RUNNABLE, group=TGRP-DistributedVersionInfoTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
        at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:382)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:593)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
        at java.lang.Thread.run(Thread.java:745)
  22) Thread[id=7500, name=qtp593071769-7500, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:546)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:609)
        at java.lang.Thread.run(Thread.java:745)
  23) Thread[id=7503, name=qtp593071769-7503, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:546)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:609)
        at java.lang.Thread.run(Thread.java:745)
  24) Thread[id=7497, name=qtp593071769-7497-acceptor-0@5431fb37-ServerConnector@778746e3{HTTP/1.1,[http/1.1]}{127.0.0.1:43197}, state=RUNNABLE, group=TGRP-DistributedVersionInfoTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
        at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:382)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:593)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
        at java.lang.Thread.run(Thread.java:745)
  25) Thread[id=7511, name=org.eclipse.jetty.server.session.HashSessionManager@5c46d9b3Timer, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
  26) Thread[id=7515, name=qtp22243454-7515, state=RUNNABLE, group=TGRP-DistributedVersionInfoTest]
        at sun.nio.ch.DevPollArrayWrapper.poll0(Native Method)
        at sun.nio.ch.DevPollArrayWrapper.poll(DevPollArrayWrapper.java:223)
        at sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:98)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:233)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:181)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:171)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
        at java.lang.Thread.run(Thread.java:745)
  27) Thread[id=7485, name=Thread-3871, state=WAITING, group=TGRP-DistributedVersionInfoTest]
        at java.lang.Object.wait(Native Method)
        at java.lang.Thread.join(Thread.java:1245)
        at java.lang.Thread.join(Thread.java:1319)
        at org.apache.zookeeper.server.NIOServerCnxnFactory.join(NIOServerCnxnFactory.java:302)
        at org.apache.solr.cloud.ZkTestServer$ZKServerMain.runFromConfig(ZkTestServer.java:310)
        at org.apache.solr.cloud.ZkTestServer$2.run(ZkTestServer.java:491)
  28) Thread[id=7501, name=qtp593071769-7501, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:546)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:609)
        at java.lang.Thread.run(Thread.java:745)
  29) Thread[id=7517, name=qtp22243454-7517, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:546)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:609)
        at java.lang.Thread.run(Thread.java:745)
  30) Thread[id=7498, name=qtp593071769-7498, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:546)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:609)
        at java.lang.Thread.run(Thread.java:745)
  31) Thread[id=7494, name=jetty-launcher-831-thread-2, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
  32) Thread[id=7487, name=SessionTracker, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]
        at java.lang.Object.wait(Native Method)
        at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146)
  33) Thread[id=7516, name=qtp22243454-7516, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:546)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:47)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:609)
        at java.lang.Thread.run(Thread.java:745)
  34) Thread[id=7496, name=qtp593071769-7496, state=RUNNABLE, group=TGRP-DistributedVersionInfoTest]
        at sun.nio.ch.DevPollArrayWrapper.poll0(Native Method)
        at sun.nio.ch.DevPollArrayWrapper.poll(DevPollArrayWrapper.java:223)
        at sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:98)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:233)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:181)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:171)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:102)
        at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:137)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
        at java.lang.Thread.run(Thread.java:745)
  35) Thread[id=7506, name=qtp22243454-7506-acceptor-0@116cf429-ServerConnector@12f8ebf0{HTTP/1.1,[http/1.1]}{127.0.0.1:37674}, state=RUNNABLE, group=TGRP-DistributedVersionInfoTest]
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
        at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:382)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:593)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([1A7ADE7FF2D37404]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.DistributedVersionInfoTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=7493, name=jetty-launcher-831-thread-1, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=7495, name=jetty-launcher-831-thread-3, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    3) Thread[id=7522, name=org.eclipse.jetty.server.session.HashSessionManager@1a8e8342Timer, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    4) Thread[id=7521, name=org.eclipse.jetty.server.session.HashSessionManager@15643d74Timer, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=7511, name=org.eclipse.jetty.server.session.HashSessionManager@5c46d9b3Timer, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    6) Thread[id=7515, name=qtp22243454-7515, state=RUNNABLE, group=TGRP-DistributedVersionInfoTest]         at sun.nio.ch.DevPollArrayWrapper.interrupt(Native Method)         at sun.nio.ch.DevPollArrayWrapper.interrupt(DevPollArrayWrapper.java:300)         at sun.nio.ch.DevPollSelectorImpl.wakeup(DevPollSelectorImpl.java:207)         at java.nio.channels.spi.AbstractSelector$1.interrupt(AbstractSelector.java:213)         at java.nio.channels.spi.AbstractSelector.begin(AbstractSelector.java:219)         at sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:233)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:181)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:171)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)         at java.lang.Thread.run(Thread.java:745)    7) Thread[id=7494, name=jetty-launcher-831-thread-2, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    8) Thread[id=7486, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-DistributedVersionInfoTest]         at sun.nio.ch.IOUtil.drain(Native Method)         at sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:109)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:178)         at java.lang.Thread.run(Thread.java:745)    9) Thread[id=7499, name=qtp1302711442-7499, state=RUNNABLE, group=TGRP-DistributedVersionInfoTest]         at sun.nio.ch.IOUtil.drain(Native Method)         at sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:109)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:233)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:181)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:171)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:102)         at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:137)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)         at java.lang.Thread.run(Thread.java:745)   10) Thread[id=7496, name=qtp593071769-7496, state=RUNNABLE, group=TGRP-DistributedVersionInfoTest]         at sun.nio.ch.IOUtil.drain(Native Method)         at sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:109)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:233)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:181)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:171)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:102)         at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:137)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)         at java.lang.Thread.run(Thread.java:745)   11) Thread[id=7529, name=Scheduler-261391908, state=WAITING, group=TGRP-DistributedVersionInfoTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=7493, name=jetty-launcher-831-thread-1, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=7495, name=jetty-launcher-831-thread-3, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   3) Thread[id=7522, name=org.eclipse.jetty.server.session.HashSessionManager@1a8e8342Timer, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   4) Thread[id=7521, name=org.eclipse.jetty.server.session.HashSessionManager@15643d74Timer, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   5) Thread[id=7511, name=org.eclipse.jetty.server.session.HashSessionManager@5c46d9b3Timer, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   6) Thread[id=7515, name=qtp22243454-7515, state=RUNNABLE, group=TGRP-DistributedVersionInfoTest]
        at sun.nio.ch.DevPollArrayWrapper.interrupt(Native Method)
        at sun.nio.ch.DevPollArrayWrapper.interrupt(DevPollArrayWrapper.java:300)
        at sun.nio.ch.DevPollSelectorImpl.wakeup(DevPollSelectorImpl.java:207)
        at java.nio.channels.spi.AbstractSelector$1.interrupt(AbstractSelector.java:213)
        at java.nio.channels.spi.AbstractSelector.begin(AbstractSelector.java:219)
        at sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:97)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:233)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:181)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:171)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
        at java.lang.Thread.run(Thread.java:745)
   7) Thread[id=7494, name=jetty-launcher-831-thread-2, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   8) Thread[id=7486, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-DistributedVersionInfoTest]
        at sun.nio.ch.IOUtil.drain(Native Method)
        at sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:109)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:178)
        at java.lang.Thread.run(Thread.java:745)
   9) Thread[id=7499, name=qtp1302711442-7499, state=RUNNABLE, group=TGRP-DistributedVersionInfoTest]
        at sun.nio.ch.IOUtil.drain(Native Method)
        at sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:109)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:233)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:181)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:171)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:102)
        at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:137)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
        at java.lang.Thread.run(Thread.java:745)
  10) Thread[id=7496, name=qtp593071769-7496, state=RUNNABLE, group=TGRP-DistributedVersionInfoTest]
        at sun.nio.ch.IOUtil.drain(Native Method)
        at sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:109)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:233)
        at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:181)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:171)
        at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:102)
        at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:137)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
        at java.lang.Thread.run(Thread.java:745)
  11) Thread[id=7529, name=Scheduler-261391908, state=WAITING, group=TGRP-DistributedVersionInfoTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([1A7ADE7FF2D37404]:0)




Build Log:
[...truncated 10980 lines...]
   [junit4] Suite: org.apache.solr.cloud.DistributedVersionInfoTest
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.DistributedVersionInfoTest_1A7ADE7FF2D37404-001/init-core-data-001
   [junit4]   2> 648621 INFO  (SUITE-DistributedVersionInfoTest-seed#[1A7ADE7FF2D37404]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
   [junit4]   2> 648622 INFO  (SUITE-DistributedVersionInfoTest-seed#[1A7ADE7FF2D37404]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 648622 INFO  (Thread-3871) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 648622 INFO  (Thread-3871) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 648722 INFO  (SUITE-DistributedVersionInfoTest-seed#[1A7ADE7FF2D37404]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:37674
   [junit4]   2> 648722 INFO  (SUITE-DistributedVersionInfoTest-seed#[1A7ADE7FF2D37404]-worker) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 648723 INFO  (SUITE-DistributedVersionInfoTest-seed#[1A7ADE7FF2D37404]-worker) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 648743 INFO  (zkCallback-832-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@33faef34 name:ZooKeeperConnection Watcher:127.0.0.1:37674 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 648743 INFO  (SUITE-DistributedVersionInfoTest-seed#[1A7ADE7FF2D37404]-worker) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 648743 INFO  (SUITE-DistributedVersionInfoTest-seed#[1A7ADE7FF2D37404]-worker) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 648743 INFO  (SUITE-DistributedVersionInfoTest-seed#[1A7ADE7FF2D37404]-worker) [    ] o.a.s.c.c.SolrZkClient makePath: /solr/solr.xml
   [junit4]   2> 648749 INFO  (jetty-launcher-831-thread-1) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 648749 INFO  (jetty-launcher-831-thread-2) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 648749 INFO  (jetty-launcher-831-thread-3) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 648751 INFO  (jetty-launcher-831-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@34d10c3c{/solr,null,AVAILABLE}
   [junit4]   2> 648753 INFO  (jetty-launcher-831-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@728a4345{/solr,null,AVAILABLE}
   [junit4]   2> 648753 INFO  (jetty-launcher-831-thread-1) [    ] o.e.j.s.ServerConnector Started ServerConnector@778746e3{HTTP/1.1,[http/1.1]}{127.0.0.1:43197}
   [junit4]   2> 648753 INFO  (jetty-launcher-831-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1bd0c532{/solr,null,AVAILABLE}
   [junit4]   2> 648753 INFO  (jetty-launcher-831-thread-1) [    ] o.e.j.s.Server Started @651919ms
   [junit4]   2> 648753 INFO  (jetty-launcher-831-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=43197}
   [junit4]   2> 648753 INFO  (jetty-launcher-831-thread-3) [    ] o.e.j.s.ServerConnector Started ServerConnector@18886709{HTTP/1.1,[http/1.1]}{127.0.0.1:34146}
   [junit4]   2> 648753 INFO  (jetty-launcher-831-thread-3) [    ] o.e.j.s.Server Started @651919ms
   [junit4]   2> 648753 INFO  (jetty-launcher-831-thread-1) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6d06d69c
   [junit4]   2> 648753 INFO  (jetty-launcher-831-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=34146}
   [junit4]   2> 648753 INFO  (jetty-launcher-831-thread-2) [    ] o.e.j.s.ServerConnector Started ServerConnector@12f8ebf0{HTTP/1.1,[http/1.1]}{127.0.0.1:37674}
   [junit4]   2> 648754 INFO  (jetty-launcher-831-thread-2) [    ] o.e.j.s.Server Started @651919ms
   [junit4]   2> 648754 INFO  (jetty-launcher-831-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37674}
   [junit4]   2> 648754 INFO  (jetty-launcher-831-thread-3) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6d06d69c
   [junit4]   2> 648754 INFO  (jetty-launcher-831-thread-3) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.DistributedVersionInfoTest_1A7ADE7FF2D37404-001/tempDir-001/node3'
   [junit4]   2> 648754 INFO  (jetty-launcher-831-thread-2) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6d06d69c
   [junit4]   2> 648754 INFO  (jetty-launcher-831-thread-3) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 648754 INFO  (jetty-launcher-831-thread-2) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.DistributedVersionInfoTest_1A7ADE7FF2D37404-001/tempDir-001/node2'
   [junit4]   2> 648754 INFO  (jetty-launcher-831-thread-3) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 648753 INFO  (jetty-launcher-831-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.DistributedVersionInfoTest_1A7ADE7FF2D37404-001/tempDir-001/node1'
   [junit4]   2> 648754 INFO  (jetty-launcher-831-thread-2) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 648754 INFO  (jetty-launcher-831-thread-2) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 648754 INFO  (jetty-launcher-831-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 648754 INFO  (jetty-launcher-831-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 648754 INFO  (jetty-launcher-831-thread-3) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 648754 

[...truncated too long message...]

  >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    5) Thread[id=7511, name=org.eclipse.jetty.server.session.HashSessionManager@5c46d9b3Timer, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    6) Thread[id=7515, name=qtp22243454-7515, state=RUNNABLE, group=TGRP-DistributedVersionInfoTest]
   [junit4]    >         at sun.nio.ch.DevPollArrayWrapper.interrupt(Native Method)
   [junit4]    >         at sun.nio.ch.DevPollArrayWrapper.interrupt(DevPollArrayWrapper.java:300)
   [junit4]    >         at sun.nio.ch.DevPollSelectorImpl.wakeup(DevPollSelectorImpl.java:207)
   [junit4]    >         at java.nio.channels.spi.AbstractSelector$1.interrupt(AbstractSelector.java:213)
   [junit4]    >         at java.nio.channels.spi.AbstractSelector.begin(AbstractSelector.java:219)
   [junit4]    >         at sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:97)
   [junit4]    >         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]    >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]    >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
   [junit4]    >         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:233)
   [junit4]    >         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:181)
   [junit4]    >         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:171)
   [junit4]    >         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    7) Thread[id=7494, name=jetty-launcher-831-thread-2, state=TIMED_WAITING, group=TGRP-DistributedVersionInfoTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    8) Thread[id=7486, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-DistributedVersionInfoTest]
   [junit4]    >         at sun.nio.ch.IOUtil.drain(Native Method)
   [junit4]    >         at sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:109)
   [junit4]    >         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]    >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]    >         at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:178)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    9) Thread[id=7499, name=qtp1302711442-7499, state=RUNNABLE, group=TGRP-DistributedVersionInfoTest]
   [junit4]    >         at sun.nio.ch.IOUtil.drain(Native Method)
   [junit4]    >         at sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:109)
   [junit4]    >         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]    >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]    >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
   [junit4]    >         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:233)
   [junit4]    >         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:181)
   [junit4]    >         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:171)
   [junit4]    >         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:102)
   [junit4]    >         at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:137)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   10) Thread[id=7496, name=qtp593071769-7496, state=RUNNABLE, group=TGRP-DistributedVersionInfoTest]
   [junit4]    >         at sun.nio.ch.IOUtil.drain(Native Method)
   [junit4]    >         at sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:109)
   [junit4]    >         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
   [junit4]    >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
   [junit4]    >         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
   [junit4]    >         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:233)
   [junit4]    >         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:181)
   [junit4]    >         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:171)
   [junit4]    >         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:102)
   [junit4]    >         at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:137)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   11) Thread[id=7529, name=Scheduler-261391908, state=WAITING, group=TGRP-DistributedVersionInfoTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([1A7ADE7FF2D37404]:0)
   [junit4] Completed [156/618 (1!)] on J0 in 85.59s, 0 tests, 3 errors <<< FAILURES!

[...truncated 1457 lines...]
BUILD FAILED
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/build.xml:740: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/build.xml:684: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/build.xml:59: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build.xml:233: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/common-build.xml:530: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/lucene/common-build.xml:1427: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/lucene/common-build.xml:984: There were test failures: 618 suites (10 ignored), 2589 tests, 3 suite-level errors, 1525 ignored (63 assumptions) [seed: 1A7ADE7FF2D37404]

Total time: 42 minutes 52 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 692 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/692/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.common.cloud.TestCollectionStateWatchers.testDeletionsTriggerWatches

Error Message:
Error from server at http://127.0.0.1:61052/solr: Could not find collection : tobedeleted

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:61052/solr: Could not find collection : tobedeleted
	at __randomizedtesting.SeedInfo.seed([EDB85F5FA5DC8E11:4F7997D027BF587C]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:606)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:413)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:366)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1228)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:998)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:934)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:166)
	at org.apache.solr.common.cloud.TestCollectionStateWatchers.testDeletionsTriggerWatches(TestCollectionStateWatchers.java:252)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 13075 lines...]
   [junit4] Suite: org.apache.solr.common.cloud.TestCollectionStateWatchers
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_EDB85F5FA5DC8E11-001/init-core-data-001
   [junit4]   2> 109560 INFO  (SUITE-TestCollectionStateWatchers-seed#[EDB85F5FA5DC8E11]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 109560 INFO  (SUITE-TestCollectionStateWatchers-seed#[EDB85F5FA5DC8E11]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 109560 INFO  (Thread-621) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 109560 INFO  (Thread-621) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 109660 INFO  (SUITE-TestCollectionStateWatchers-seed#[EDB85F5FA5DC8E11]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:45168
   [junit4]   2> 109661 INFO  (SUITE-TestCollectionStateWatchers-seed#[EDB85F5FA5DC8E11]-worker) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 109661 INFO  (SUITE-TestCollectionStateWatchers-seed#[EDB85F5FA5DC8E11]-worker) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 109665 INFO  (zkCallback-181-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5edfcc6e name:ZooKeeperConnection Watcher:127.0.0.1:45168 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 109665 INFO  (SUITE-TestCollectionStateWatchers-seed#[EDB85F5FA5DC8E11]-worker) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 109666 INFO  (SUITE-TestCollectionStateWatchers-seed#[EDB85F5FA5DC8E11]-worker) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 109666 INFO  (SUITE-TestCollectionStateWatchers-seed#[EDB85F5FA5DC8E11]-worker) [    ] o.a.s.c.c.SolrZkClient makePath: /solr/solr.xml
   [junit4]   2> 109676 INFO  (jetty-launcher-180-thread-2) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 109676 INFO  (jetty-launcher-180-thread-1) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 109677 INFO  (jetty-launcher-180-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@51ba6fe{/solr,null,AVAILABLE}
   [junit4]   2> 109677 INFO  (jetty-launcher-180-thread-4) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 109678 INFO  (jetty-launcher-180-thread-3) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 109680 INFO  (jetty-launcher-180-thread-4) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1419efe{/solr,null,AVAILABLE}
   [junit4]   2> 109680 INFO  (jetty-launcher-180-thread-4) [    ] o.e.j.s.ServerConnector Started ServerConnector@5ee9e7f8{HTTP/1.1,[http/1.1]}{127.0.0.1:33701}
   [junit4]   2> 109680 INFO  (jetty-launcher-180-thread-4) [    ] o.e.j.s.Server Started @115263ms
   [junit4]   2> 109680 INFO  (jetty-launcher-180-thread-4) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33701}
   [junit4]   2> 109681 INFO  (jetty-launcher-180-thread-4) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6d06d69c
   [junit4]   2> 109681 INFO  (jetty-launcher-180-thread-4) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_EDB85F5FA5DC8E11-001/tempDir-001/node4'
   [junit4]   2> 109681 INFO  (jetty-launcher-180-thread-4) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 109681 INFO  (jetty-launcher-180-thread-4) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 109681 INFO  (jetty-launcher-180-thread-4) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 109683 INFO  (jetty-launcher-180-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6a4bf8fd{/solr,null,AVAILABLE}
   [junit4]   2> 109684 INFO  (jetty-launcher-180-thread-1) [    ] o.e.j.s.ServerConnector Started ServerConnector@7c0b0a85{HTTP/1.1,[http/1.1]}{127.0.0.1:44147}
   [junit4]   2> 109684 INFO  (jetty-launcher-180-thread-1) [    ] o.e.j.s.Server Started @115267ms
   [junit4]   2> 109684 INFO  (jetty-launcher-180-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44147}
   [junit4]   2> 109684 INFO  (jetty-launcher-180-thread-1) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6d06d69c
   [junit4]   2> 109684 INFO  (jetty-launcher-180-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_EDB85F5FA5DC8E11-001/tempDir-001/node1'
   [junit4]   2> 109684 INFO  (jetty-launcher-180-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 109685 INFO  (jetty-launcher-180-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 109685 INFO  (jetty-launcher-180-thread-1) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 109685 INFO  (jetty-launcher-180-thread-1) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 109687 INFO  (jetty-launcher-180-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@19d4dd2a{/solr,null,AVAILABLE}
   [junit4]   2> 109688 INFO  (jetty-launcher-180-thread-3) [    ] o.e.j.s.ServerConnector Started ServerConnector@1cff4f74{HTTP/1.1,[http/1.1]}{127.0.0.1:40929}
   [junit4]   2> 109688 INFO  (jetty-launcher-180-thread-3) [    ] o.e.j.s.Server Started @115270ms
   [junit4]   2> 109688 INFO  (jetty-launcher-180-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40929}
   [junit4]   2> 109688 INFO  (jetty-launcher-180-thread-3) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6d06d69c
   [junit4]   2> 109688 INFO  (jetty-launcher-180-thread-3) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_EDB85F5FA5DC8E11-001/tempDir-001/node3'
   [junit4]   2> 109688 INFO  (jetty-launcher-180-thread-3) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 109688 INFO  (jetty-launcher-180-thread-3) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 109689 INFO  (jetty-launcher-180-thread-2) [    ] o.e.j.s.ServerConnector Started ServerConnector@7538a16d{HTTP/1.1,[http/1.1]}{127.0.0.1:61052}
   [junit4]   2> 109689 INFO  (jetty-launcher-180-thread-3) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 109689 INFO  (jetty-launcher-180-thread-2) [    ] o.e.j.s.Server Started @115272ms
   [junit4]   2> 109689 INFO  (jetty-launcher-180-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=61052}
   [junit4]   2> 109689 INFO  (jetty-launcher-180-thread-2) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6d06d69c
   [junit4]   2> 109689 INFO  (jetty-launcher-180-thread-2) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_EDB85F5FA5DC8E11-001/tempDir-001/node2'
   [junit4]   2> 109689 INFO  (jetty-launcher-180-thread-2) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 109689 INFO  (jetty-launcher-180-thread-2) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 109689 INFO  (jetty-launcher-180-thread-3) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 109690 INFO  (jetty-launcher-180-thread-2) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 109691 INFO  (jetty-launcher-180-thread-2) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 109692 INFO  (jetty-launcher-180-thread-4) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 109694 INFO  (zkCallback-183-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@69640513 name:ZooKeeperConnection Watcher:127.0.0.1:45168/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 109694 INFO  (jetty-launcher-180-thread-1) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 109694 INFO  (jetty-launcher-180-thread-1) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 109696 INFO  (zkCallback-185-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4b3da560 name:ZooKeeperConnection Watcher:127.0.0.1:45168/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 109696 INFO  (jetty-launcher-180-thread-2) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 109696 INFO  (jetty-launcher-180-thread-2) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 109697 INFO  (zkCallback-182-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@17e71fcf name:ZooKeeperConnection Watcher:127.0.0.1:45168/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 109698 INFO  (jetty-launcher-180-thread-4) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 109698 INFO  (jetty-launcher-180-thread-4) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 109699 INFO  (jetty-launcher-180-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 109699 INFO  (zkCallback-184-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6fbf9be8 name:ZooKeeperConnection Watcher:127.0.0.1:45168/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 109699 INFO  (jetty-launcher-180-thread-3) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 109699 INFO  (jetty-launcher-180-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 109699 INFO  (jetty-launcher-180-thread-3) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 109700 INFO  (jetty-launcher-180-thread-4) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 109707 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x155b3b2c7d80004, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 109708 INFO  (jetty-launcher-180-thread-4) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_EDB85F5FA5DC8E11-001/tempDir-001/node4/.
   [junit4]   2> 109708 INFO  (jetty-launcher-180-thread-4) [    ] o.a.s.c.CoreContainer New CoreContainer 218215017
   [junit4]   2> 109708 INFO  (jetty-launcher-180-thread-4) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_EDB85F5FA5DC8E11-001/tempDir-001/node4]
   [junit4]   2> 109709 WARN  (jetty-launcher-180-thread-4) [    ] o.a.s.c.CoreContainer Couldn't add files from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_EDB85F5FA5DC8E11-001/tempDir-001/node4/lib to classpath: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_EDB85F5FA5DC8E11-001/tempDir-001/node4/lib
   [junit4]   2> 109709 INFO  (jetty-launcher-180-thread-4) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,
   [junit4]   2> 109708 INFO  (jetty-launcher-180-thread-2) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_EDB85F5FA5DC8E11-001/tempDir-001/node2/.
   [junit4]   2> 109710 INFO  (jetty-launcher-180-thread-2) [    ] o.a.s.c.CoreContainer New CoreContainer 1118599053
   [junit4]   2> 109710 INFO  (jetty-launcher-180-thread-2) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_EDB85F5FA5DC8E11-001/tempDir-001/node2]
   [junit4]   2> 109710 WARN  (jetty-launcher-180-thread-2) [    ] o.a.s.c.CoreContainer Couldn't add files from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_EDB85F5FA5DC8E11-001/tempDir-001/node2/lib to classpath: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_EDB85F5FA5DC8E11-001/tempDir-001/node2/lib
   [junit4]   2> 109710 INFO  (jetty-launcher-180-thread-2) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,
   [junit4]   2> 109710 INFO  (jetty-launcher-180-thread-4) [    ] o.a.s.u.UpdateShardHandler Created UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&maxConnections=100000&maxConnectionsPerHost=100000
   [junit4]   2> 109710 INFO  (jetty-launcher-180-thread-4) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 109710 INFO  (jetty-launcher-180-thread-4) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 109710 INFO  (jetty-launcher-180-thread-4) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45168/solr
   [junit4]   2> 109710 INFO  (jetty-launcher-180-thread-4) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 109710 INFO  (jetty-launcher-180-thread-4) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 109710 INFO  (jetty-launcher-180-thread-4) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 109711 INFO  (jetty-launcher-180-thread-2) [    ] o.a.s.u.UpdateShardHandler Created UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&maxConnections=100000&maxConnectionsPerHost=100000
   [junit4]   2> 109711 INFO  (jetty-launcher-180-thread-2) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 109711 INFO  (jetty-launcher-180-thread-2) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 109711 INFO  (jetty-launcher-180-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45168/solr
   [junit4]   2> 109711 INFO  (jetty-launcher-180-thread-2) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 109711 INFO  (jetty-launcher-180-thread-2) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 109712 INFO  (jetty-launcher-180-thread-2) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 109715 INFO  (jetty-launcher-180-thread-1) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_EDB85F5FA5DC8E11-001/tempDir-001/node1/.
   [junit4]   2> 109715 INFO  (jetty-launcher-180-thread-1) [    ] o.a.s.c.CoreContainer New CoreContainer 728899538
   [junit4]   2> 109715 INFO  (jetty-launcher-180-thread-1) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_EDB85F5FA5DC8E11-001/tempDir-001/node1]
   [junit4]   2> 109715 WARN  (jetty-launcher-180-thread-1) [    ] o.a.s.c.CoreContainer Couldn't add files from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_EDB85F5FA5DC8E11-001/tempDir-001/node1/lib to classpath: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_EDB85F5FA5DC8E11-001/tempDir-001/node1/lib
   [junit4]   2> 109715 INFO  (jetty-launcher-180-thread-1) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,
   [junit4]   2> 109716 INFO  (jetty-launcher-180-thread-1) [    ] o.a.s.u.UpdateShardHandler Created UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&maxConnections=100000&maxConnectionsPerHost=100000
   [junit4]   2> 109716 INFO  (jetty-launcher-180-thread-1) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 109716 INFO  (jetty-launcher-180-thread-1) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 109716 INFO  (jetty-launcher-180-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45168/solr
   [junit4]   2> 109716 INFO  (jetty-launcher-180-thread-1) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 109716 INFO  (jetty-launcher-180-thread-1) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 109716 INFO  (zkCallback-191-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5b7bee83 name:ZooKeeperConnection Watcher:127.0.0.1:45168 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 109717 INFO  (jetty-launcher-180-thread-1) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 109717 INFO  (jetty-launcher-180-thread-2) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 109717 INFO  (jetty-launcher-180-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 109717 INFO  (jetty-launcher-180-thread-2) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 109718 INFO  (zkCallback-188-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@753b67c0 name:ZooKeeperConnection Watcher:127.0.0.1:45168 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 109722 INFO  (jetty-launcher-180-thread-3) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_EDB85F5FA5DC8E11-001/tempDir-001/node3/.
   [junit4]   2> 109722 INFO  (jetty-launcher-180-thread-3) [    ] o.a.s.c.CoreContainer New CoreContainer 1680614953
   [junit4]   2> 109722 INFO  (jetty-launcher-180-thread-3) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_EDB85F5FA5DC8E11-001/tempDir-001/node3]
   [junit4]   2> 109722 WARN  (jetty-launcher-180-thread-3) [    ] o.a.s.c.CoreContainer Couldn't add files from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_EDB85F5FA5DC8E11-001/tempDir-001/node3/lib to classpath: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_EDB85F5FA5DC8E11-001/tempDir-001/node3/lib
   [junit4]   2> 109722 INFO  (jetty-launcher-180-thread-3) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,
   [junit4]   2> 109723 INFO  (jetty-launcher-180-thread-4) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 109723 INFO  (jetty-launcher-180-thread-4) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 109724 INFO  (jetty-launcher-180-thread-3) [    ] o.a.s.u.UpdateShardHandler Created UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&maxConnections=100000&maxConnectionsPerHost=100000
   [junit4]   2> 109724 INFO  (jetty-launcher-180-thread-3) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 109724 INFO  (jetty-launcher-180-thread-3) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 109724 INFO  (jetty-launcher-180-thread-3) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45168/solr
   [junit4]   2> 109724 INFO  (jetty-launcher-180-thread-3) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 109724 INFO  (jetty-launcher-180-thread-3) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 109724 INFO  (jetty-launcher-180-thread-3) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 109728 INFO  (zkCallback-194-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@64b8095b name:ZooKeeperConnection Watcher:127.0.0.1:45168 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 109728 INFO  (jetty-launcher-180-thread-1) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 109728 INFO  (jetty-launcher-180-thread-1) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 109728 INFO  (jetty-launcher-180-thread-2) [n:127.0.0.1:61052_solr    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$33/1574173981@3b3f3fde
   [junit4]   2> 109729 INFO  (jetty-launcher-180-thread-2) [n:127.0.0.1:61052_solr    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 109730 INFO  (zkCallback-197-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@31ad8d4c name:ZooKeeperConnection Watcher:127.0.0.1:45168 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 109730 INFO  (jetty-launcher-180-thread-3) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 109730 INFO  (jetty-launcher-180-thread-3) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 109730 INFO  (jetty-launcher-180-thread-4) [n:127.0.0.1:33701_solr    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$33/1574173981@6168b96e
   [junit4]   2> 109731 INFO  (jetty-launcher-180-thread-4) [n:127.0.0.1:33701_solr    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 109733 INFO  (jetty-launcher-180-thread-1) [n:127.0.0.1:44147_solr    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$33/1574173981@d9e2e2d
   [junit4]   2> 109733 INFO  (jetty-launcher-180-thread-1) [n:127.0.0.1:44147_solr    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 109734 INFO  (zkCallback-199-thread-1-processing-n:127.0.0.1:33701_solr) [n:127.0.0.1:33701_solr    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@577417a1 name:ZooKeeperConnection Watcher:127.0.0.1:45168/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 109734 INFO  (jetty-launcher-180-thread-3) [n:127.0.0.1:40929_solr    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$33/1574173981@30a5ff85
   [junit4]   2> 109735 INFO  (jetty-launcher-180-thread-4) [n:127.0.0.1:33701_solr    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 109735 INFO  (zkCallback-198-thread-1-processing-n:127.0.0.1:61052_solr) [n:127.0.0.1:61052_solr    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@56563393 name:ZooKeeperConnection Watcher:127.0.0.1:45168/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 109735 INFO  (jetty-launcher-180-thread-3) [n:127.0.0.1:40929_solr    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 109735 INFO  (jetty-launcher-180-thread-2) [n:127.0.0.1:61052_solr    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 109738 INFO  (jetty-launcher-180-thread-4) [n:127.0.0.1:33701_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 109738 INFO  (zkCallback-200-thread-1-processing-n:127.0.0.1:44147_solr) [n:127.0.0.1:44147_solr    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7d819a7d name:ZooKeeperConnection Watcher:127.0.0.1:45168/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 109738 INFO  (jetty-launcher-180-thread-1) [n:127.0.0.1:44147_solr    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 109738 INFO  (zkCallback-201-thread-1-processing-n:127.0.0.1:40929_solr) [n:127.0.0.1:40929_solr    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5c0fa8a name:ZooKeeperConnection Watcher:127.0.0.1:45168/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 109739 INFO  (jetty-launcher-180-thread-3) [n:127.0.0.1:40929_solr    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 109742 INFO  (jetty-launcher-180-thread-1) [n:127.0.0.1:44147_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 109743 INFO  (jetty-launcher-180-thread-2) [n:127.0.0.1:61052_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 109744 INFO  (jetty-launcher-180-thread-3) [n:127.0.0.1:40929_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 109745 INFO  (jetty-launcher-180-thread-1) [n:127.0.0.1:44147_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 109746 INFO  (jetty-launcher-180-thread-4) [n:127.0.0.1:33701_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 109746 INFO  (jetty-launcher-180-thread-2) [n:127.0.0.1:61052_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 109748 INFO  (jetty-launcher-180-thread-3) [n:127.0.0.1:40929_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 109750 INFO  (jetty-launcher-180-thread-2) [n:127.0.0.1:61052_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 109751 INFO  (jetty-launcher-180-thread-4) [n:127.0.0.1:33701_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 109752 INFO  (jetty-launcher-180-thread-1) [n:127.0.0.1:44147_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 109754 INFO  (jetty-launcher-180-thread-2) [n:127.0.0.1:61052_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 109756 INFO  (jetty-launcher-180-thread-2) [n:127.0.0.1:61052_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 109756 INFO  (jetty-launcher-180-thread-1) [n:127.0.0.1:44147_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 109757 INFO  (jetty-launcher-180-thread-4) [n:127.0.0.1:33701_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 109759 INFO  (jetty-launcher-180-thread-2) [n:127.0.0.1:61052_solr    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 109759 INFO  (jetty-launcher-180-thread-3) [n:127.0.0.1:40929_solr    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 109760 INFO  (jetty-launcher-180-thread-1) [n:127.0.0.1:44147_solr    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 109760 INFO  (jetty-launcher-180-thread-2) [n:127.0.0.1:61052_solr    ] o.a.s.c.c.SolrZkClient makePath: /collections
   [junit4]   2> 109761 INFO  (jetty-launcher-180-thread-4) [n:127.0.0.1:33701_solr    ] o.a.s.c.c.SolrZkClient makePath: /collections
   [junit4]   2> 109761 INFO  (jetty-launcher-180-thread-3) [n:127.0.0.1:40929_solr    ] o.a.s.c.c.SolrZkClient makePath: /collections
   [junit4]   2> 109762 INFO  (jetty-launcher-180-thread-1) [n:127.0.0.1:44147_solr    ] o.a.s.c.c.SolrZkClient makePath: /collections
   [junit4]   2> 109763 INFO  (jetty-launcher-180-thread-2) [n:127.0.0.1:61052_solr    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 109763 INFO  (jetty-launcher-180-thread-3) [n:127.0.0.1:40929_solr    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 109763 INFO  (jetty-launcher-180-thread-4) [n:127.0.0.1:33701_solr    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 109764 INFO  (jetty-launcher-180-thread-2) [n:127.0.0.1:61052_solr    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 109765 INFO  (jetty-launcher-180-thread-4) [n:127.0.0.1:33701_solr    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 109765 INFO  (jetty-launcher-180-thread-1) [n:127.0.0.1:44147_solr    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 109766 INFO  (jetty-launcher-180-thread-3) [n:127.0.0.1:40929_solr    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 109766 INFO  (jetty-launcher-180-thread-2) [n:127.0.0.1:61052_solr    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 109766 INFO  (jetty-launcher-180-thread-4) [n:127.0.0.1:33701_solr    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 109767 INFO  (jetty-launcher-180-thread-1) [n:127.0.0.1:44147_solr    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 109767 INFO  (jetty-launcher-180-thread-3) [n:127.0.0.1:40929_solr    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 109768 INFO  (jetty-launcher-180-thread-2) [n:127.0.0.1:61052_solr    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 109768 INFO  (jetty-launcher-180-thread-1) [n:127.0.0.1:44147_solr    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 109769 INFO  (jetty-launcher-180-thread-3) [n:127.0.0.1:40929_solr    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 109769 INFO  (jetty-launcher-180-thread-2) [n:127.0.0.1:61052_solr    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 109769 INFO  (jetty-launcher-180-thread-4) [n:127.0.0.1:33701_solr    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 109769 INFO  (jetty-launcher-180-thread-1) [n:127.0.0.1:44147_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 109769 INFO  (jetty-launcher-180-thread-2) [n:127.0.0.1:61052_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 109769 INFO  (jetty-launcher-180-thread-3) [n:127.0.0.1:40929_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 109770 INFO  (jetty-launcher-180-thread-4) [n:127.0.0.1:33701_solr    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 109772 INFO  (jetty-launcher-180-thread-4) [n:127.0.0.1:33701_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 109773 INFO  (jetty-launcher-180-thread-2) [n:127.0.0.1:61052_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 109775 INFO  (jetty-launcher-180-thread-2) [n:127.0.0.1:61052_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 109775 INFO  (jetty-launcher-180-thread-3) [n:127.0.0.1:40929_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 109776 INFO  (jetty-launcher-180-thread-1) [n:127.0.0.1:44147_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 109777 INFO  (jetty-launcher-180-thread-3) [n:127.0.0.1:40929_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 109777 INFO  (jetty-launcher-180-thread-2) [n:127.0.0.1:61052_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 109777 INFO  (jetty-launcher-180-thread-1) [n:127.0.0.1:44147_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 109778 INFO  (jetty-launcher-180-thread-2) [n:127.0.0.1:61052_solr    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96180547496706057-127.0.0.1:61052_solr-n_0000000001
   [junit4]   2> 109778 INFO  (jetty-launcher-180-thread-3) [n:127.0.0.1:40929_solr    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96180547496706060-127.0.0.1:40929_solr-n_0000000000
   [junit4]   2> 109778 INFO  (jetty-launcher-180-thread-1) [n:127.0.0.1:44147_solr    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96180547496706059-127.0.0.1:44147_solr-n_0000000002
   [junit4]   2> 109779 INFO  (jetty-launcher-180-thread-3) [n:127.0.0.1:40929_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:40929_solr
   [junit4]   2> 109779 INFO  (jetty-launcher-180-thread-3) [n:127.0.0.1:40929_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 109779 INFO  (jetty-launcher-180-thread-2) [n:127.0.0.1:61052_solr    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/96180547496706060-127.0.0.1:40929_solr-n_0000000000 to know if I could be the leader
   [junit4]   2> 109779 INFO  (jetty-launcher-180-thread-2) [n:127.0.0.1:61052_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:61052_solr
   [junit4]   2> 109779 INFO  (jetty-launcher-180-thread-2) [n:127.0.0.1:61052_solr    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:61052_solr
   [junit4]   2> 109780 INFO  (jetty-launcher-180-thread-1) [n:127.0.0.1:44147_solr    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/96180547496706057-127.0.0.1:61052_solr-n_0000000001 to know if I could be the leader
   [junit4]   2> 109780 INFO  (jetty-launcher-180-thread-1) [n:127.0.0.1:44147_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44147_solr
   [junit4]   2> 109780 INFO  (jetty-launcher-180-thread-1) [n:127.0.0.1:44147_solr    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:44147_solr
   [junit4]   2> 109780 INFO  (jetty-launcher-180-thread-4) [n:127.0.0.1:33701_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 109781 INFO  (zkCallback-200-thread-1-processing-n:127.0.0.1:44147_solr) [n:127.0.0.1:44147_solr    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 109781 INFO  (jetty-launcher-180-thread-4) [n:127.0.0.1:33701_solr    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96180547496706058-127.0.0.1:33701_solr-n_0000000003
   [junit4]   2> 109781 INFO  (zkCallback-199-thread-1-processing-n:127.0.0.1:33701_solr) [n:127.0.0.1:33701_solr    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 109783 INFO  (zkCallback-201-thread-1-processing-n:127.0.0.1:40929_solr) [n:127.0.0.1:40929_solr    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 109788 INFO  (zkCallback-198-thread-1-processing-n:127.0.0.1:61052_solr) [n:127.0.0.1:61052_solr    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 109790 INFO  (zkCallback-201-thread-2-processing-n:127.0.0.1:40929_solr) [n:127.0.0.1:40929_solr    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 109790 INFO  (jetty-launcher-180-thread-3) [n:127.0.0.1:40929_solr    ] o.a.s.c.Overseer Overseer (id=96180547496706060-127.0.0.1:40929_solr-n_0000000000) starting
   [junit4]   2> 109793 INFO  (zkCallback-198-thread-1-processing-n:127.0.0.1:61052_solr) [n:127.0.0.1:61052_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 109793 INFO  (zkCallback-201-thread-1-processing-n:127.0.0.1:40929_solr) [n:127.0.0.1:40929_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 109794 INFO  (zkCallback-200-thread-1-processing-n:127.0.0.1:44147_solr) [n:127.0.0.1:44147_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 109794 INFO  (zkCallback-199-thread-1-processing-n:127.0.0.1:33701_solr) [n:127.0.0.1:33701_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 109794 INFO  (zkCallback-200-thread-2-processing-n:127.0.0.1:44147_solr) [n:127.0.0.1:44147_solr    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 109795 INFO  (jetty-launcher-180-thread-4) [n:127.0.0.1:33701_solr    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/96180547496706059-127.0.0.1:44147_solr-n_0000000002 to know if I could be the leader
   [junit4]   2> 109795 INFO  (zkCallback-200-thread-2-processing-n:127.0.0.1:44147_solr) [n:127.0.0.1:44147_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 109795 INFO  (jetty-launcher-180-thread-4) [n:127.0.0.1:33701_solr    ] o.a.s.c.ZkController Publish node=127.0.0.1:33701_solr as DOWN
   [junit4]   2> 109796 INFO  (jetty-launcher-180-thread-4) [n:127.0.0.1:33701_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33701_solr
   [junit4]   2> 109796 INFO  (jetty-launcher-180-thread-4) [n:127.0.0.1:33701_solr    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:33701_solr
   [junit4]   2> 109797 INFO  (jetty-launcher-180-thread-2) [n:127.0.0.1:61052_solr    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 109797 INFO  (jetty-launcher-180-thread-2) [n:127.0.0.1:61052_solr    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 109798 INFO  (zkCallback-199-thread-1-processing-n:127.0.0.1:33701_solr) [n:127.0.0.1:33701_solr    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 109798 INFO  (jetty-launcher-180-thread-2) [n:127.0.0.1:61052_solr    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_EDB85F5FA5DC8E11-001/tempDir-001/node2/.
   [junit4]   2> 109798 INFO  (zkCallback-199-thread-1-processing-n:127.0.0.1:33701_solr) [n:127.0.0.1:33701_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 109798 INFO  (jetty-launcher-180-thread-2) [n:127.0.0.1:61052_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions
   [junit4]   2> 109798 INFO  (zkCallback-198-thread-1-processing-n:127.0.0.1:61052_solr) [n:127.0.0.1:61052_solr    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 109799 INFO  (zkCallback-198-thread-1-processing-n:127.0.0.1:61052_solr) [n:127.0.0.1:61052_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 109799 INFO  (zkCallback-200-thread-2-processing-n:127.0.0.1:44147_solr) [n:127.0.0.1:44147_solr    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 109800 INFO  (jetty-launcher-180-thread-2) [n:127.0.0.1:61052_solr    ] o.a.s.s.SolrDispatchFilter user.dir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J0
   [junit4]   2> 109800 INFO  (zkCallback-200-thread-2-processing-n:127.0.0.1:44147_solr) [n:127.0.0.1:44147_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 109805 INFO  (jetty-launcher-180-thread-2) [n:127.0.0.1:61052_solr    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 109811 INFO  (zkCallback-201-thread-2-processing-n:127.0.0.1:40929_solr) [n:127.0.0.1:40929_solr    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 109812 INFO  (zkCallback-201-thread-2-processing-n:127.0.0.1:40929_solr) [n:127.0.0.1:40929_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 109813 INFO  (jetty-launcher-180-thread-3) [n:127.0.0.1:40929_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 109824 INFO  (jetty-launcher-180-thread-4) [n:127.0.0.1:33701_solr    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 109824 INFO  (jetty-launcher-180-thread-4) [n:127.0.0.1:33701_solr    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 109826 INFO  (jetty-launcher-180-thread-4) [n:127.0.0.1:33701_solr    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_EDB85F5FA5DC8E11-001/tempDir-001/node4/.
   [junit4]   2> 109827 INFO  (jetty-launcher-180-thread-4) [n:127.0.0.1:33701_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions
   [junit4]   2> 109830 INFO  (jetty-launcher-180-thread-4) [n:127.0.0.1:33701_solr    ] o.a.s.s.SolrDispatchFilter user.dir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J0
   [junit4]   2> 109830 INFO  (jetty-launcher-180-thread-4) [n:127.0.0.1:33701_solr    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 109831 INFO  (jetty-launcher-180-thread-3) [n:127.0.0.1:40929_solr    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 109831 INFO  (OverseerCollectionConfigSetProcessor-96180547496706060-127.0.0.1:40929_solr-n_0000000000) [n:127.0.0.1:40929_solr    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 109832 INFO  (OverseerStateUpdate-96180547496706060-127.0.0.1:40929_solr-n_0000000000) [n:127.0.0.1:40929_solr    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 109832 INFO  (jetty-launcher-180-thread-3) [n:127.0.0.1:40929_solr    ] o.a.s.c.ZkController Publish node=127.0.0.1:40929_solr as DOWN
   [junit4]   2> 109836 INFO  (jetty-launcher-180-thread-3) [n:127.0.0.1:40929_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40929_solr
   [junit4]   2> 109836 INFO  (jetty-launcher-180-thread-3) [n:127.0.0.1:40929_solr    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:40929_solr
   [junit4]   2> 109837 INFO  (OverseerStateUpdate-96180547496706060-127.0.0.1:40929_solr-n_0000000000) [n:127.0.0.1:40929_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (3)
   [junit4]   2> 109838 INFO  (zkCallback-200-thread-2-processing-n:127.0.0.1:44147_solr) [n:127.0.0.1:44147_solr    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [3])
   [junit4]   2> 109838 INFO  (zkCallback-198-thread-1-processing-n:127.0.0.1:61052_solr) [n:127.0.0.1:61052_solr    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [3])
   [junit4]   2> 109839 INFO  (zkCallback-200-thread-2-processing-n:127.0.0.1:44147_solr) [n:127.0.0.1:44147_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 109839 INFO  (zkCallback-198-thread-1-processing-n:127.0.0.1:61052_solr) [n:127.0.0.1:61052_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 109839 INFO  (zkCallback-199-thread-1-processing-n:127.0.0.1:33701_solr) [n:127.0.0.1:33701_solr    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [3])
   [junit4]   2> 109839 INFO  (zkCallback-201-thread-2-processing-n:127.0.0.1:40929_solr) [n:127.0.0.1:40929_solr    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [3])
   [junit4]   2> 109840 INFO  (zkCallback-199-thread-1-processing-n:127.0.0.1:33701_solr) [n:127.0.0.1:33701_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 109843 INFO  (zkCallback-201-thread-2-processing-n:127.0.0.1:40929_solr) [n:127.0.0.1:40929_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 109845 INFO  (OverseerStateUpdate-96180547496706060-127.0.0.1:40929_solr-n_0000000000) [n:127.0.0.1:40929_solr    ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:33701_solr"} current state version: 0
   [junit4]   2> 109845 INFO  (OverseerStateUpdate-96180547496706060-127.0.0.1:40929_solr-n_0000000000) [n:127.0.0.1:40929_solr    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:33701_solr
   [junit4]   2> 109851 INFO  (OverseerStateUpdate-96180547496706060-127.0.0.1:40929_solr-n_0000000000) [n:127.0.0.1:40929_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:40929_solr"} current state version: 0
   [junit4]   2> 109851 INFO  (OverseerStateUpdate-96180547496706060-127.0.0.1:40929_solr-n_0000000000) [n:127.0.0.1:40929_solr    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:40929_solr
   [junit4]   2> 109877 INFO  (jetty-launcher-180-thread-3) [n:127.0.0.1:40929_solr    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 109877 INFO  (jetty-launcher-180-thread-3) [n:127.0.0.1:40929_solr    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 109878 INFO  (jetty-launcher-180-thread-3) [n:127.0.0.1:40929_solr    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_EDB85F5FA5DC8E11-001/tempDir-001/node3/.
   [junit4]   2> 109879 INFO  (jetty-launcher-180-thread-3) [n:127.0.0.1:40929_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions
   [junit4]   2> 109881 INFO  (jetty-launcher-180-thread-3) [n:127.0.0.1:40929_solr    ] o.a.s.s.SolrDispatchFilter user.dir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J0
   [junit4]   2> 109881 INFO  (jetty-launcher-180-thread-3) [n:127.0.0.1:40929_solr    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 109884 INFO  (jetty-launcher-180-thread-1) [n:127.0.0.1:44147_solr    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 109884 INFO  (jetty-launcher-180-thread-1) [n:127.0.0.1:44147_solr    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 109886 INFO  (jetty-launcher-180-thread-1) [n:127.0.0.1:44147_solr    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_EDB85F5FA5DC8E11-001/tempDir-001/node1/.
   [junit4]   2> 109887 INFO  (jetty-launcher-180-thread-1) [n:127.0.0.1:44147_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions
   [junit4]   2> 109888 INFO  (jetty-launcher-180-thread-1) [n:127.0.0.1:44147_solr    ] o.a.s.s.SolrDispatchFilter user.dir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J0
   [junit4]   2> 109888 INFO  (jetty-launcher-180-thread-1) [n:127.0.0.1:44147_solr    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 109888 INFO  (SUITE-TestCollectionStateWatchers-seed#[EDB85F5FA5DC8E11]-worker) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 109889 INFO  (SUITE-TestCollectionStateWatchers-seed#[EDB85F5FA5DC8E11]-worker) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 109892 INFO  (zkCallback-206-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3a3002e5 name:ZooKeeperConnection Watcher:127.0.0.1:45168 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 109892 INFO  (SUITE-TestCollectionStateWatchers-seed#[EDB85F5FA5DC8E11]-worker) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 109892 INFO  (SUITE-TestCollectionStateWatchers-seed#[EDB85F5FA5DC8E11]-worker) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 109898 INFO  (SUITE-TestCollectionStateWatchers-seed#[EDB85F5FA5DC8E11]-worker) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 109898 INFO  (SUITE-TestCollectionStateWatchers-seed#[EDB85F5FA5DC8E11]-worker) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 109903 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x155b3b2c7d8000d, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 109904 INFO  (zkCallback-209-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@110ea9de name:ZooKeeperConnection Watcher:127.0.0.1:45168/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 109904 INFO  (SUITE-TestCollectionStateWatchers-seed#[EDB85F5FA5DC8E11]-worker) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 109904 INFO  (SUITE-TestCollectionStateWatchers-seed#[EDB85F5FA5DC8E11]-worker) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 109904 INFO  (SUITE-TestCollectionStateWatchers-seed#[EDB85F5FA5DC8E11]-worker) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 109907 INFO  (SUITE-TestCollectionStateWatchers-seed#[EDB85F5FA5DC8E11]-worker) [    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 109908 INFO  (SUITE-TestCollectionStateWatchers-seed#[EDB85F5FA5DC8E11]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 109910 INFO  (SUITE-TestCollectionStateWatchers-seed#[EDB85F5FA5DC8E11]-worker) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/config/solrconfig.xml
   [junit4]   2> 109914 INFO  (SUITE-TestCollectionStateWatchers-seed#[EDB85F5FA5DC8E11]-worker) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/config/solrconfig-sql.xml
   [junit4]   2> 109917 INFO  (SUITE-TestCollectionStateWatchers-seed#[EDB85F5FA5DC8E11]-worker) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/config/schema-replication1.xml
   [junit4]   2> 109919 INFO  (SUITE-TestCollectionStateWatchers-seed#[EDB85F5FA5DC8E11]-worker) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/config/solrconfig-managed-schema.xml
   [junit4]   2> 109922 INFO  (SUITE-TestCollectionStateWatchers-seed#[EDB85F5FA5DC8E11]-worker) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/config/schema-sql.xml
   [junit4]   2> 109925 INFO  (SUITE-TestCollectionStateWatchers-seed#[EDB85F5FA5DC8E11]-worker) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/config/solrconfig-slave1.xml
   [junit4]   2> 109928 INFO  (SUITE-TestCollectionStateWatchers-seed#[EDB85F5FA5DC8E11]-worker) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/config/schema.xml
   [junit4]   2> 110043 INFO  (TEST-TestCollectionStateWatchers.testCanWaitForNonexistantCollection-seed#[EDB85F5FA5DC8E11]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testCanWaitForNonexistantCollection
   [junit4]   2> 110043 INFO  (TEST-TestCollectionStateWatchers.testCanWaitForNonexistantCollection-seed#[EDB85F5FA5DC8E11]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 110044 INFO  (TEST-TestCollectionStateWatchers.testCanWaitForNonexistantCollection-seed#[EDB85F5FA5DC8E11]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 110047 INFO  (zkCallback-210-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@f3c1c3c name:ZooKeeperConnection Watcher:127.0.0.1:45168 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 110047 INFO  (TEST-TestCollectionStateWatchers.testCanWaitForNonexistantCollection-seed#[EDB85F5FA5DC8E11]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 110047 INFO  (TEST-TestCollectionStateWatchers.testCanWaitForNonexistantCollection-seed#[EDB85F5FA5DC8E11]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 110053 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x155b3b2c7d8000f, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 110055 INFO  (qtp2028339803-1486) [n:127.0.0.1:61052_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params async=9ece4e6f-f7c3-49cb-a762-6c3eed5d7806&replicationFactor=1&collection.configName=config&name=delayed&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 110058 INFO  (qtp2028339803-1486) [n:127.0.0.1:61052_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=9ece4e6f-f7c3-49cb-a762-6c3eed5d7806&replicationFactor=1&collection.configName=config&name=delayed&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=3
   [junit4]   2> 110064 INFO  (qtp2028339803-1479) [n:127.0.0.1:61052_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=9ece4e6f-f7c3-49cb-a762-6c3eed5d7806&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 110065 INFO  (OverseerCollectionConfigSetProcessor-96180547496706060-127.0.0.1:40929_solr-n_0000000000) [n:127.0.0.1:40929_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running/mn-9ece4e6f-f7c3-49cb-a762-6c3eed5d7806
   [junit4]   2> 110065 INFO  (backgroundWatchers-179-thread-1) [    ] o.a.s.c.c.ZkStateReader Deleting data for [delayed]
   [junit4]   2> 110068 INFO  (qtp2028339803-1479) [n:127.0.0.1:61052_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=9ece4e6f-f7c3-49cb-a762-6c3eed5d7806&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=4
   [junit4]   2> 110068 INFO  (OverseerCollectionConfigSetProcessor-96180547496706060-127.0.0.1:40929_solr-n_0000000000) [n:127.0.0.1:40929_solr    ] o.a.s.c.OverseerTaskProcessor Overseer Collection Message Handler: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
   [junit4]   2>   "name":"delayed",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "collection.configName":"config",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "async":"9ece4e6f-f7c3-49cb-a762-6c3eed5d7806",
   [junit4]   2>   "operation":"create"}
   [junit4]   2> 110068 INFO  (OverseerThreadFactory-763-thread-1-processing-n:127.0.0.1:40929_solr) [n:127.0.0.1:40929_solr    ] o.a.s.c.OverseerCollectionMessageHandler OverseerCollectionMessageHandler.processMessage : create , {
   [junit4]   2>   "name":"delayed",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "collection.configName":"config",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "async":"9ece4e6f-f7c3-49cb-a762-6c3eed5d7806",
   [junit4]   2>   "operation":"create"}
   [junit4]   2> 110068 INFO  (OverseerThreadFactory-763-thread-1-processing-n:127.0.0.1:40929_solr) [n:127.0.0.1:40929_solr    ] o.a.s.c.OverseerCollectionMessageHandler Create collection delayed
   [junit4]   2> 110069 INFO  (OverseerThreadFactory-763-thread-1-processing-n:127.0.0.1:40929_solr) [n:127.0.0.1:40929_solr    ] o.a.s.c.OverseerCollectionMessageHandler creating collections conf node /collections/delayed 
   [junit4]   2> 110070 INFO  (OverseerThreadFactory-763-thread-1-processing-n:127.0.0.1:40929_solr) [n:127.0.0.1:40929_solr    ] o.a.s.c.c.SolrZkClient makePath: /collections/delayed
   [junit4]   2> 110072 INFO  (zkCallback-199-thread-1-processing-n:127.0.0.1:33701_solr) [n:127.0.0.1:33701_solr    ] o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections], has occurred - updating...
   [junit4]   2> 110075 INFO  (OverseerStateUpdate-96180547496706060-127.0.0.1:40929_solr-n_0000000000) [n:127.0.0.1:40929_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "name":"delayed",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "collection.configName":"config",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "async":"9ece4e6f-f7c3-49cb-a762-6c3eed5d7806",
   [junit4]   2>   "operation":"create"} current state version: 0
   [junit4]   2> 110075 INFO  (OverseerStateUpdate-96180547496706060-127.0.0.1:40929_solr-n_0000000000) [n:127.0.0.1:40929_solr    ] o.a.s.c.o.ClusterStateMutator building a new cName: delayed
   [junit4]   2> 110077 INFO  (zkCallback-200-thread-2-processing-n:127.0.0.1:44147_solr) [n:127.0.0.1:44147_solr    ] o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections], has occurred - updating...
   [junit4]   2> 110078 INFO  (OverseerStateUpdate-96180547496706060-127.0.0.1:40929_solr-n_0000000000) [n:127.0.0.1:40929_solr    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/delayed/state.json
   [junit4]   2> 110082 INFO  (zkCallback-209-thread-1) [    ] o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections], has occurred - updating...
   [junit4]   2> 110083 INFO  (zkCallback-209-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeCreated path:/collections/delayed/state.json] for collection [delayed] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 110083 INFO  (zkCallback-198-thread-1-processing-n:127.0.0.1:61052_solr) [n:127.0.0.1:61052_solr    ] o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections], has occurred - updating...
   [junit4]   2> 110083 INFO  (zkCallback-209-thread-2) [    ] o.a.s.c.c.ZkStateReader Add data for [delayed] ver [0]
   [junit4]   2> 110084 INFO  (zkCallback-201-thread-2-processing-n:127.0.0.1:40929_solr) [n:127.0.0.1:40929_solr    ] o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections], has occurred - updating...
   [junit4]   2> 110176 INFO  (OverseerThreadFactory-763-thread-1-processing-n:127.0.0.1:40929_solr) [n:127.0.0.1:40929_solr    ] o.a.s.c.OverseerCollectionMessageHandler Creating SolrCores for new collection delayed, shardNames [shard1] , replicationFactor : 1
   [junit4]   2> 110176 INFO  (OverseerThreadFactory-763-thread-1-processing-n:127.0.0.1:40929_solr) [n:127.0.0.1:40929_solr    ] o.a.s.c.OverseerCollectionMessageHandler Creating core delayed_shard1_replica1 as part of shard shard1 of collection delayed on 127.0.0.1:40929_solr
   [junit4]   2> 110179 INFO  (qtp1879451596-1510) [n:127.0.0.1:40929_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396&qt=/admin/cores&collection.configName=config&name=delayed_shard1_replica1&action=CREATE&numShards=1&collection=delayed&shard=shard1&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 110180 INFO  (parallelCoreAdminExecutor-764-thread-1-processing-n:127.0.0.1:40929_solr 9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396 CREATE) [n:127.0.0.1:40929_solr    ] o.a.s.h.a.CoreAdminOperation core create command async=9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396&qt=/admin/cores&collection.configName=config&name=delayed_shard1_replica1&action=CREATE&numShards=1&collection=delayed&shard=shard1&wt=javabin&version=2
   [junit4]   2> 110180 INFO  (parallelCoreAdminExecutor-764-thread-1-processing-n:127.0.0.1:40929_solr 9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396 CREATE) [n:127.0.0.1:40929_solr    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=delayed_shard1_replica1, collection.configName=config, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=delayed, numShards=1, dataDir=data/, shard=shard1}
   [junit4]   2> 110181 INFO  (qtp1879451596-1510) [n:127.0.0.1:40929_solr    ] o.a.s.h.a.CoreAdminOperation Checking request status for : 9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396
   [junit4]   2> 110181 INFO  (qtp1879451596-1510) [n:127.0.0.1:40929_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 110183 INFO  (parallelCoreAdminExecutor-764-thread-1-processing-n:127.0.0.1:40929_solr 9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396 CREATE) [n:127.0.0.1:40929_solr c:delayed s:shard1  x:delayed_shard1_replica1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 110185 INFO  (parallelCoreAdminExecutor-764-thread-1-processing-n:127.0.0.1:40929_solr 9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396 CREATE) [n:127.0.0.1:40929_solr c:delayed s:shard1  x:delayed_shard1_replica1] o.a.s.c.ZkController Registering watch for collection delayed
   [junit4]   2> 110185 INFO  (parallelCoreAdminExecutor-764-thread-1-processing-n:127.0.0.1:40929_solr 9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396 CREATE) [n:127.0.0.1:40929_solr c:delayed s:shard1  x:delayed_shard1_replica1] o.a.s.c.c.ZkStateReader Add data for [delayed] ver [0]
   [junit4]   2> 110186 INFO  (parallelCoreAdminExecutor-764-thread-1-processing-n:127.0.0.1:40929_solr 9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396 CREATE) [n:127.0.0.1:40929_solr c:delayed s:shard1  x:delayed_shard1_replica1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 110186 INFO  (OverseerStateUpdate-96180547496706060-127.0.0.1:40929_solr-n_0000000000) [n:127.0.0.1:40929_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"delayed_shard1_replica1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:40929/solr",
   [junit4]   2>   "node_name":"127.0.0.1:40929_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"delayed",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 110186 INFO  (OverseerStateUpdate-96180547496706060-127.0.0.1:40929_solr-n_0000000000) [n:127.0.0.1:40929_solr    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"delayed_shard1_replica1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:40929/solr",
   [junit4]   2>   "node_name":"127.0.0.1:40929_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"delayed",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 110291 INFO  (OverseerStateUpdate-96180547496706060-127.0.0.1:40929_solr-n_0000000000) [n:127.0.0.1:40929_solr    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/delayed/state.json version: 0
   [junit4]   2> 110292 INFO  (zkCallback-209-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delayed/state.json] for collection [delayed] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 110292 INFO  (zkCallback-201-thread-1-processing-n:127.0.0.1:40929_solr) [n:127.0.0.1:40929_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delayed/state.json] for collection [delayed] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 110292 INFO  (zkCallback-201-thread-1-processing-n:127.0.0.1:40929_solr) [n:127.0.0.1:40929_solr    ] o.a.s.c.c.ZkStateReader Updating data for [delayed] from [0] to [1]
   [junit4]   2> 110293 INFO  (zkCallback-209-thread-2) [    ] o.a.s.c.c.ZkStateReader Updating data for [delayed] from [0] to [1]
   [junit4]   2> 111070 INFO  (qtp2028339803-1480) [n:127.0.0.1:61052_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=9ece4e6f-f7c3-49cb-a762-6c3eed5d7806&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 111073 INFO  (qtp2028339803-1480) [n:127.0.0.1:61052_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=9ece4e6f-f7c3-49cb-a762-6c3eed5d7806&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2
   [junit4]   2> 111183 INFO  (qtp1879451596-1512) [n:127.0.0.1:40929_solr    ] o.a.s.h.a.CoreAdminOperation Checking request status for : 9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396
   [junit4]   2> 111184 INFO  (qtp1879451596-1512) [n:127.0.0.1:40929_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 111186 INFO  (parallelCoreAdminExecutor-764-thread-1-processing-n:127.0.0.1:40929_solr 9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396 CREATE) [n:127.0.0.1:40929_solr c:delayed s:shard1  x:delayed_shard1_replica1] o.a.s.c.ZkController waiting to find shard id in clusterstate for delayed_shard1_replica1
   [junit4]   2> 111186 INFO  (parallelCoreAdminExecutor-764-thread-1-processing-n:127.0.0.1:40929_solr 9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396 CREATE) [n:127.0.0.1:40929_solr c:delayed s:shard1  x:delayed_shard1_replica1] o.a.s.c.ZkController Check for collection zkNode:delayed
   [junit4]   2> 111187 INFO  (parallelCoreAdminExecutor-764-thread-1-processing-n:127.0.0.1:40929_solr 9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396 CREATE) [n:127.0.0.1:40929_solr c:delayed s:shard1  x:delayed_shard1_replica1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 111187 INFO  (parallelCoreAdminExecutor-764-thread-1-processing-n:127.0.0.1:40929_solr 9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396 CREATE) [n:127.0.0.1:40929_solr c:delayed s:shard1  x:delayed_shard1_replica1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/delayed]
   [junit4]   2> 111188 INFO  (parallelCoreAdminExecutor-764-thread-1-processing-n:127.0.0.1:40929_solr 9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396 CREATE) [n:127.0.0.1:40929_solr c:delayed s:shard1  x:delayed_shard1_replica1] o.a.s.c.c.ZkStateReader path=[/collections/delayed] [configName]=[config] specified config exists in ZooKeeper
   [junit4]   2> 111188 INFO  (parallelCoreAdminExecutor-764-thread-1-processing-n:127.0.0.1:40929_solr 9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396 CREATE) [n:127.0.0.1:40929_solr c:delayed s:shard1  x:delayed_shard1_replica1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_EDB85F5FA5DC8E11-001/tempDir-001/node3/delayed_shard1_replica1'
   [junit4]   2> 111188 INFO  (parallelCoreAdminExecutor-764-thread-1-processing-n:127.0.0.1:40929_solr 9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396 CREATE) [n:127.0.0.1:40929_solr c:delayed s:shard1  x:delayed_shard1_replica1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 111188 INFO  (parallelCoreAdminExecutor-764-thread-1-processing-n:127.0.0.1:40929_solr 9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396 CREATE) [n:127.0.0.1:40929_solr c:delayed s:shard1  x:delayed_shard1_replica1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 111191 INFO  (parallelCoreAdminExecutor-764-thread-1-processing-n:127.0.0.1:40929_solr 9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396 CREATE) [n:127.0.0.1:40929_solr c:delayed s:shard1  x:delayed_shard1_replica1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 111195 INFO  (parallelCoreAdminExecutor-764-thread-1-processing-n:127.0.0.1:40929_solr 9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396 CREATE) [n:127.0.0.1:40929_solr c:delayed s:shard1  x:delayed_shard1_replica1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 111197 INFO  (parallelCoreAdminExecutor-764-thread-1-processing-n:127.0.0.1:40929_solr 9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396 CREATE) [n:127.0.0.1:40929_solr c:delayed s:shard1  x:delayed_shard1_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 111203 INFO  (parallelCoreAdminExecutor-764-thread-1-processing-n:127.0.0.1:40929_solr 9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396 CREATE) [n:127.0.0.1:40929_solr c:delayed s:shard1  x:delayed_shard1_replica1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 111207 INFO  (parallelCoreAdminExecutor-764-thread-1-processing-n:127.0.0.1:40929_solr 9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396 CREATE) [n:127.0.0.1:40929_solr c:delayed s:shard1  x:delayed_shard1_replica1] o.a.s.s.IndexSchema [delayed_shard1_replica1] Schema name=test
   [junit4]   2> 111275 INFO  (parallelCoreAdminExecutor-764-thread-1-processing-n:127.0.0.1:40929_solr 9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396 CREATE) [n:127.0.0.1:40929_solr c:delayed s:shard1  x:delayed_shard1_replica1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 111276 INFO  (parallelCoreAdminExecutor-764-thread-1-processing-n:127.0.0.1:40929_solr 9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396 CREATE) [n:127.0.0.1:40929_solr c:delayed s:shard1  x:delayed_shard1_replica1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 111277 WARN  (parallelCoreAdminExecutor-764-thread-1-processing-n:127.0.0.1:40929_solr 9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396 CREATE) [n:127.0.0.1:40929_solr c:delayed s:shard1  x:delayed_shard1_replica1] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
   [junit4]   2> 111277 WARN  (parallelCoreAdminExecutor-764-thread-1-processing-n:127.0.0.1:40929_solr 9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396 CREATE) [n:127.0.0.1:40929_solr c:delayed s:shard1  x:delayed_shard1_replica1] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3)
   [junit4]   2> 111279 INFO  (parallelCoreAdminExecutor-764-thread-1-processing-n:127.0.0.1:40929_solr 9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396 CREATE) [n:127.0.0.1:40929_solr c:delayed s:shard1  x:delayed_shard1_replica1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/config', cwd=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J0
   [junit4]   2> 111279 INFO  (parallelCoreAdminExecutor-764-thread-1-processing-n:127.0.0.1:40929_solr 9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396 CREATE) [n:127.0.0.1:40929_solr c:delayed s:shard1  x:delayed_shard1_replica1] o.a.s.c.CoreContainer Creating SolrCore 'delayed_shard1_replica1' using configuration from collection delayed
   [junit4]   2> 111280 INFO  (parallelCoreAdminExecutor-764-thread-1-processing-n:127.0.0.1:40929_solr 9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396 CREATE) [n:127.0.0.1:40929_solr c:delayed s:shard1 r:core_node1 x:delayed_shard1_replica1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 111280 INFO  (parallelCoreAdminExecutor-764-thread-1-processing-n:127.0.0.1:40929_solr 9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396 CREATE) [n:127.0.0.1:40929_solr c:delayed s:shard1 r:core_node1 x:delayed_shard1_replica1] o.a.s.c.SolrCore [[delayed_shard1_replica1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_EDB85F5FA5DC8E11-001/tempDir-001/node3/delayed_shard1_replica1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_EDB85F5FA5DC8E11-001/tempDir-001/node3/./delayed_shard1_replica1/data/]
   [junit4]   2> 111280 INFO  (parallelCoreAdminExecutor-764-thread-1-processing-n:127.0.0.1:40929_solr 9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396 CREATE) [n:127.0.0.1:40929_solr c:delayed s:shard1 r:core_node1 x:delayed_shard1_replica1] o.a.s.c.SolrCore JMX monitoring not detected for core: delayed_shard1_replica1
   [junit4]   2> 111280 INFO  (parallelCoreAdminExecutor-764-thread-1-processing-n:127.0.0.1:40929_solr 9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396 CREATE) [n:127.0.0.1:40929_solr c:delayed s:shard1 r:core_node1 x:delayed_shard1_replica1] o.a.s.c.CachingDirectoryFactory return new directory for /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_EDB85F5FA5DC8E11-001/tempDir-001/node3/./delayed_shard1_replica1/data
   [junit4]   2> 111280 INFO  (parallelCoreAdminExecutor-764-thread-1-processing-n:127.0.0.1:40929_solr 9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396 CREATE) [n:127.0.0.1:40929_solr c:delayed s:shard1 r:core_node1 x:delayed_shard1_replica1] o.a.s.c.SolrCore New index directory detected: old=null new=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_EDB85F5FA5DC8E11-001/tempDir-001/node3/./delayed_shard1_replica1/data/index/
   [junit4]   2> 111280 WARN  (parallelCoreAdminExecutor-764-thread-1-processing-n:127.0.0.1:40929_solr 9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396 CREATE) [n:127.0.0.1:40929_solr c:delayed s:shard1 r:core_node1 x:delayed_shard1_replica1] o.a.s.c.SolrCore [delayed_shard1_replica1] Solr index directory '/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_EDB85F5FA5DC8E11-001/tempDir-001/node3/./delayed_shard1_replica1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 111281 INFO  (parallelCoreAdminExecutor-764-thread-1-processing-n:127.0.0.1:40929_solr 9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396 CREATE) [n:127.0.0.1:40929_solr c:delayed s:shard1 r:core_node1 x:delayed_shard1_replica1] o.a.s.c.CachingDirectoryFactory return new directory for /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_EDB85F5FA5DC8E11-001/tempDir-001/node3/./delayed_shard1_replica1/data/index
   [junit4]   2> 111281 INFO  (parallelCoreAdminExecutor-764-thread-1-processing-n:127.0.0.1:40929_solr 9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396 CREATE) [n:127.0.0.1:40929_solr c:delayed s:shard1 r:core_node1 x:delayed_shard1_replica1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@4c243fd lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@56fe8203),segFN=segments_1,generation=1}
   [junit4]   2> 111281 INFO  (parallelCoreAdminExecutor-764-thread-1-processing-n:127.0.0.1:40929_solr 9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396 CREATE) [n:127.0.0.1:40929_solr c:delayed s:shard1 r:core_node1 x:delayed_shard1_replica1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 111282 INFO  (OldIndexDirectoryCleanupThreadForCore-delayed_shard1_replica1) [n:127.0.0.1:40929_solr c:delayed s:shard1 r:core_node1 x:delayed_shard1_replica1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core delayed_shard1_replica1 in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_EDB85F5FA5DC8E11-001/tempDir-001/node3/./delayed_shard1_replica1/data/
   [junit4]   2> 111283 WARN  (OldIndexDirectoryCleanupThreadForCore-delayed_shard1_replica1) [n:127.0.0.1:40929_solr c:delayed s:shard1 r:core_node1 x:delayed_shard1_replica1] o.a.s.c.DirectoryFactory /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_EDB85F5FA5DC8E11-001/tempDir-001/node3/./delayed_shard1_replica1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 111286 INFO  (parallelCoreAdminExecutor-764-thread-1-processing-n:127.0.0.1:40929_solr 9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396 CREATE) [n:127.0.0.1:40929_solr c:delayed s:shard1 r:core_node1 x:delayed_shard1_replica1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 111287 INFO  (parallelCoreAdminExecutor-764-thread-1-processing-n:127.0.0.1:40929_solr 9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396 CREATE) [n:127.0.0.1:40929_solr c:delayed s:shard1 r:core_node1 x:delayed_shard1_replica1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 111288 INFO  (parallelCoreAdminExecutor-764-thread-1-processing-n:127.0.0.1:40929_solr 9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396 CREATE) [n:127.0.0.1:40929_solr c:delayed s:shard1 r:core_node1 x:delayed_shard1_replica1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 111289 INFO  (parallelCoreAdminExecutor-764-thread-1-processing-n:127.0.0.1:40929_solr 9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396 CREATE) [n:127.0.0.1:40929_solr c:delayed s:shard1 r:core_node1 x:delayed_shard1_replica1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 111290 INFO  (parallelCoreAdminExecutor-764-thread-1-processing-n:127.0.0.1:40929_solr 9ece4e6f-f7c3-49cb-a762-6c3eed5d7806741713195322396 CREATE) [n:127.0.0.1:40929_solr c:delayed s:shard1 r:core_node1 x:delayed_shard1_replica1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 111292 INFO  (p

[...truncated too long message...]

null,UNAVAILABLE}
   [junit4]   2> 163224 INFO  (zkCallback-201-thread-11-processing-n:127.0.0.1:40929_solr) [n:127.0.0.1:40929_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
   [junit4]   2> 163226 INFO  (zkCallback-233-thread-1-processing-n:127.0.0.1:46830_solr) [n:127.0.0.1:46830_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
   [junit4]   2> 163226 INFO  (zkCallback-201-thread-12-processing-n:127.0.0.1:40929_solr) [n:127.0.0.1:40929_solr    ] o.a.s.c.c.ZkStateReader Updating data for [delayed] from [12] to [13]
   [junit4]   2> 163228 INFO  (zkCallback-233-thread-1-processing-n:127.0.0.1:46830_solr) [n:127.0.0.1:46830_solr    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 163229 INFO  (zkCallback-201-thread-12-processing-n:127.0.0.1:40929_solr) [n:127.0.0.1:40929_solr    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 163229 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x155b3b2c7d8001e, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 163230 INFO  (zkCallback-201-thread-12-processing-n:127.0.0.1:40929_solr) [n:127.0.0.1:40929_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 163231 INFO  (zkCallback-201-thread-12-processing-n:127.0.0.1:40929_solr) [n:127.0.0.1:40929_solr    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 164693 WARN  (zkCallback-222-thread-1-processing-n:127.0.0.1:33701_solr) [n:127.0.0.1:33701_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 164693 INFO  (jetty-launcher-180-thread-4) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2290cb43{/solr,null,UNAVAILABLE}
   [junit4]   2> 164726 WARN  (zkCallback-233-thread-2-processing-n:127.0.0.1:46830_solr) [n:127.0.0.1:46830_solr    ] o.a.s.c.LeaderElector 
   [junit4]   2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election/96180547496706060-127.0.0.1:40929_solr-n_0000000000
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:349)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:346)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:346)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:152)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:56)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:348)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3.lambda$process$0(SolrZkClient.java:267)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 164733 WARN  (zkCallback-233-thread-1-processing-n:127.0.0.1:46830_solr) [n:127.0.0.1:46830_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 164733 WARN  (zkCallback-201-thread-12-processing-n:127.0.0.1:40929_solr) [n:127.0.0.1:40929_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 164733 INFO  (jetty-launcher-180-thread-2) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@17e1e66f{/solr,null,UNAVAILABLE}
   [junit4]   2> 164733 INFO  (jetty-launcher-180-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@19d4dd2a{/solr,null,UNAVAILABLE}
   [junit4]   2> 164737 INFO  (SUITE-TestCollectionStateWatchers-seed#[EDB85F5FA5DC8E11]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:45168 45168
   [junit4]   2> 164846 INFO  (Thread-621) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:45168 45168
   [junit4]   2> 164847 WARN  (Thread-621) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	8	/solr/aliases.json
   [junit4]   2> 	8	/solr/clusterprops.json
   [junit4]   2> 	7	/solr/security.json
   [junit4]   2> 	7	/solr/configs/config
   [junit4]   2> 	3	/solr/collections/stateformat1/state.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	15	/solr/collections/falsepredicate/state.json
   [junit4]   2> 	12	/solr/collections/waitforstate/state.json
   [junit4]   2> 	8	/solr/clusterstate.json
   [junit4]   2> 	7	/solr/collections/testcollection/state.json
   [junit4]   2> 	3	/solr/overseer_elect/election/96180547496706060-127.0.0.1:40929_solr-n_0000000000
   [junit4]   2> 	3	/solr/collections/stateformat1/state.json
   [junit4]   2> 	2	/solr/collections/currentstate/state.json
   [junit4]   2> 	2	/solr/overseer_elect/election/96180547496706067-127.0.0.1:61052_solr-n_0000000004
   [junit4]   2> 	2	/solr/collections/delayed/state.json
   [junit4]   2> 	2	/solr/overseer_elect/election/96180547496706059-127.0.0.1:44147_solr-n_0000000002
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	53	/solr/overseer/queue
   [junit4]   2> 	35	/solr/overseer/collection-queue-work
   [junit4]   2> 	18	/solr/overseer/queue-work
   [junit4]   2> 	8	/solr/collections
   [junit4]   2> 	5	/solr/live_nodes
   [junit4]   2> 
   [junit4]   2> 164848 INFO  (SUITE-TestCollectionStateWatchers-seed#[EDB85F5FA5DC8E11]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_EDB85F5FA5DC8E11-001
   [junit4]   2> Jul 04, 2016 2:19:01 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene62): {}, docValues:{}, maxPointsInLeafNode=1004, maxMBSortInHeap=7.0581820053334035, sim=RandomSimilarity(queryNorm=true,coord=crazy): {}, locale=pt-BR, timezone=SystemV/EST5EDT
   [junit4]   2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_92 (64-bit)/cpus=3,threads=1,free=173762096,total=260046848
   [junit4]   2> NOTE: All tests run in this JVM: [TestCollectionAdminRequest, ShardParamsTest, TestToleratedUpdateError, ConcurrentUpdateSolrClientTest, TestJavaBinCodec, TestSpellCheckResponse, JettyWebappTest, TestCoreAdmin, ConcatOperationTest, SolrDocumentTest, TestClusteringResponse, CloudSolrClientTest, FieldAnalysisResponseTest, TestRetryUtil, CollectionAdminRequestRequiredParamsTest, TestHash, SchemaTest, SolrExampleStreamingTest, TestSolrProperties, TestCollectionStateWatchers]
   [junit4] Completed [70/83 (1!)] on J0 in 56.08s, 9 tests, 1 error <<< FAILURES!

[...truncated 49 lines...]
BUILD FAILED
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/build.xml:740: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/build.xml:684: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/build.xml:59: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build.xml:246: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/common-build.xml:530: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/lucene/common-build.xml:1427: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/lucene/common-build.xml:984: There were test failures: 83 suites, 645 tests, 1 error, 2 ignored [seed: EDB85F5FA5DC8E11]

Total time: 79 minutes 16 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any