You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2017/05/22 18:58:17 UTC

[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1308 - Still Unstable

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1308/

4 tests failed.
FAILED:  org.apache.solr.ltr.TestLTRQParserPlugin.ltrMoreResultsThanReRankedTest

Error Message:
mismatch: '0.09271725'!='0.105360515' @ response/docs/[3]/score

Stack Trace:
java.lang.RuntimeException: mismatch: '0.09271725'!='0.105360515' @ response/docs/[3]/score
	at __randomizedtesting.SeedInfo.seed([2D6C3E0309DABA87:E4148F64D615ED46]:0)
	at org.apache.solr.util.RestTestBase.assertJQ(RestTestBase.java:248)
	at org.apache.solr.util.RestTestBase.assertJQ(RestTestBase.java:192)
	at org.apache.solr.ltr.TestLTRQParserPlugin.ltrMoreResultsThanReRankedTest(TestLTRQParserPlugin.java:94)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.ltr.TestParallelWeightCreation.testLTRScoringQueryParallelWeightCreationResultOrder

Error Message:
mismatch: '3'!='4' @ response/docs/[0]/id

Stack Trace:
java.lang.RuntimeException: mismatch: '3'!='4' @ response/docs/[0]/id
	at __randomizedtesting.SeedInfo.seed([2D6C3E0309DABA87:48201E63C3B8DD6F]:0)
	at org.apache.solr.util.RestTestBase.assertJQ(RestTestBase.java:248)
	at org.apache.solr.util.RestTestBase.assertJQ(RestTestBase.java:192)
	at org.apache.solr.ltr.TestParallelWeightCreation.testLTRScoringQueryParallelWeightCreationResultOrder(TestParallelWeightCreation.java:45)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.ltr.TestSelectiveWeightCreation.testSelectiveWeightsRequestFeaturesFromDifferentStore

Error Message:
mismatch: '3'!='4' @ response/docs/[0]/id

Stack Trace:
java.lang.RuntimeException: mismatch: '3'!='4' @ response/docs/[0]/id
	at __randomizedtesting.SeedInfo.seed([2D6C3E0309DABA87:705D29C684E6CDEC]:0)
	at org.apache.solr.util.RestTestBase.assertJQ(RestTestBase.java:248)
	at org.apache.solr.util.RestTestBase.assertJQ(RestTestBase.java:192)
	at org.apache.solr.ltr.TestSelectiveWeightCreation.testSelectiveWeightsRequestFeaturesFromDifferentStore(TestSelectiveWeightCreation.java:230)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.ltr.feature.TestFieldLengthFeature.testRanking

Error Message:
mismatch: '8'!='1' @ response/docs/[0]/id

Stack Trace:
java.lang.RuntimeException: mismatch: '8'!='1' @ response/docs/[0]/id
	at __randomizedtesting.SeedInfo.seed([2D6C3E0309DABA87:B25A979D91A0A548]:0)
	at org.apache.solr.util.RestTestBase.assertJQ(RestTestBase.java:248)
	at org.apache.solr.util.RestTestBase.assertJQ(RestTestBase.java:192)
	at org.apache.solr.ltr.feature.TestFieldLengthFeature.testRanking(TestFieldLengthFeature.java:117)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 20166 lines...]
   [junit4] Suite: org.apache.solr.ltr.feature.TestFieldLengthFeature
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/contrib/solr-ltr/test/J0/temp/solr.ltr.feature.TestFieldLengthFeature_2D6C3E0309DABA87-001/init-core-data-001
   [junit4]   2> 16854 WARN  (SUITE-TestFieldLengthFeature-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=3 numCloses=3
   [junit4]   2> 16854 INFO  (SUITE-TestFieldLengthFeature-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields
   [junit4]   2> 16856 INFO  (SUITE-TestFieldLengthFeature-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 16859 INFO  (SUITE-TestFieldLengthFeature-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore
   [junit4]   2> 16873 INFO  (SUITE-TestFieldLengthFeature-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 16896 INFO  (SUITE-TestFieldLengthFeature-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.s.IndexSchema [null] Schema name=example
   [junit4]   2> 16912 INFO  (SUITE-TestFieldLengthFeature-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.s.IndexSchema Loaded schema example/1.5 with uniqueid field id
   [junit4]   2> 16960 WARN  (SUITE-TestFieldLengthFeature-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.m.r.SolrJmxReporter No JMX server found. Not exposing Solr metrics via JMX.
   [junit4]   2> 16960 WARN  (SUITE-TestFieldLengthFeature-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.m.r.SolrJmxReporter No JMX server found. Not exposing Solr metrics via JMX.
   [junit4]   2> 16960 WARN  (SUITE-TestFieldLengthFeature-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.m.r.SolrJmxReporter No JMX server found. Not exposing Solr metrics via JMX.
   [junit4]   2> 16984 INFO  (coreLoadExecutor-97-thread-1) [    x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 16998 INFO  (coreLoadExecutor-97-thread-1) [    x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=example
   [junit4]   2> 17011 INFO  (coreLoadExecutor-97-thread-1) [    x:collection1] o.a.s.s.IndexSchema Loaded schema example/1.5 with uniqueid field id
   [junit4]   2> 17019 INFO  (coreLoadExecutor-97-thread-1) [    x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/contrib/ltr/src/test-files/solr/collection1, trusted=true
   [junit4]   2> 17020 WARN  (coreLoadExecutor-97-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter No JMX server found. Not exposing Solr metrics via JMX.
   [junit4]   2> 17020 INFO  (coreLoadExecutor-97-thread-1) [    x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 17020 INFO  (coreLoadExecutor-97-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/contrib/ltr/src/test-files/solr/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/contrib/solr-ltr/test/J0/temp/solr.ltr.feature.TestFieldLengthFeature_2D6C3E0309DABA87-001/init-core-data-001/]
   [junit4]   2> 17255 WARN  (coreLoadExecutor-97-thread-1) [    x:collection1] o.a.s.c.RequestHandlers no default request handler is registered (either '/select' or 'standard')
   [junit4]   2> 17266 INFO  (coreLoadExecutor-97-thread-1) [    x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 17266 INFO  (coreLoadExecutor-97-thread-1) [    x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 17267 INFO  (coreLoadExecutor-97-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 15000ms; 
   [junit4]   2> 17267 INFO  (coreLoadExecutor-97-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: if uncommited for 1000ms; 
   [junit4]   2> 17272 INFO  (coreLoadExecutor-97-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@14824ca5[collection1] main]
   [junit4]   2> 17273 WARN  (coreLoadExecutor-97-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Cannot write to config directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/contrib/ltr/src/test-files/solr/collection1/conf; switching to use InMemory storage instead.
   [junit4]   2> 17273 INFO  (coreLoadExecutor-97-thread-1) [    x:collection1] o.a.s.r.RestManager Registered ManagedResource impl org.apache.solr.ltr.store.rest.ManagedFeatureStore for path /schema/feature-store
   [junit4]   2> 17273 WARN  (coreLoadExecutor-97-thread-1) [    x:collection1] o.a.s.r.ManagedResource No stored data found for /schema/feature-store
   [junit4]   2> 17273 INFO  (coreLoadExecutor-97-thread-1) [    x:collection1] o.a.s.l.s.r.ManagedFeatureStore ------ managed feature ~ loading ------
   [junit4]   2> 17273 INFO  (coreLoadExecutor-97-thread-1) [    x:collection1] o.a.s.r.ManagedResource Notified 1 observers of /schema/feature-store
   [junit4]   2> 17273 INFO  (coreLoadExecutor-97-thread-1) [    x:collection1] o.a.s.r.RestManager Registered new managed resource /schema/feature-store
   [junit4]   2> 17273 INFO  (coreLoadExecutor-97-thread-1) [    x:collection1] o.a.s.r.RestManager Registered ManagedResource impl org.apache.solr.ltr.store.rest.ManagedModelStore for path /schema/model-store
   [junit4]   2> 17273 WARN  (coreLoadExecutor-97-thread-1) [    x:collection1] o.a.s.r.ManagedResource No stored data found for /schema/model-store
   [junit4]   2> 17273 INFO  (coreLoadExecutor-97-thread-1) [    x:collection1] o.a.s.l.s.r.ManagedModelStore INIT model store
   [junit4]   2> 17273 INFO  (coreLoadExecutor-97-thread-1) [    x:collection1] o.a.s.l.s.r.ManagedModelStore ------ managed models ~ loading ------
   [junit4]   2> 17273 INFO  (coreLoadExecutor-97-thread-1) [    x:collection1] o.a.s.r.ManagedResource Notified 1 observers of /schema/model-store
   [junit4]   2> 17273 INFO  (coreLoadExecutor-97-thread-1) [    x:collection1] o.a.s.r.RestManager Registered new managed resource /schema/model-store
   [junit4]   2> 17274 INFO  (coreLoadExecutor-97-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 17274 INFO  (coreLoadExecutor-97-thread-1) [    x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1568123138902851584
   [junit4]   2> 17275 INFO  (searcherExecutor-98-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@14824ca5[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 17286 INFO  (SUITE-TestFieldLengthFeature-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore end
   [junit4]   2> 17292 INFO  (SUITE-TestFieldLengthFeature-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore
   [junit4]   2> 17292 INFO  (SUITE-TestFieldLengthFeature-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore end
   [junit4]   2> 17293 INFO  (SUITE-TestFieldLengthFeature-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/contrib/solr-ltr/test/J0/temp/solr.ltr.feature.TestFieldLengthFeature_2D6C3E0309DABA87-001/tempDir-003/cores/core
   [junit4]   2> 17308 INFO  (SUITE-TestFieldLengthFeature-seed#[2D6C3E0309DABA87]-worker) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 17382 INFO  (SUITE-TestFieldLengthFeature-seed#[2D6C3E0309DABA87]-worker) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6e7c7c09{/solr,null,AVAILABLE}
   [junit4]   2> 17383 INFO  (SUITE-TestFieldLengthFeature-seed#[2D6C3E0309DABA87]-worker) [    ] o.e.j.s.AbstractConnector Started ServerConnector@4c4bb677{SSL,[ssl, http/1.1]}{127.0.0.1:55085}
   [junit4]   2> 17383 INFO  (SUITE-TestFieldLengthFeature-seed#[2D6C3E0309DABA87]-worker) [    ] o.e.j.s.Server Started @19541ms
   [junit4]   2> 17383 INFO  (SUITE-TestFieldLengthFeature-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=55085, configSetBaseDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/contrib/solr-ltr/test/J0/temp/solr.ltr.feature.TestFieldLengthFeature_2D6C3E0309DABA87-001/tempDir-001, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/contrib/solr-ltr/test/J0/temp/solr.ltr.feature.TestFieldLengthFeature_2D6C3E0309DABA87-001/tempDir-003/cores}
   [junit4]   2> 17384 ERROR (SUITE-TestFieldLengthFeature-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 17384 INFO  (SUITE-TestFieldLengthFeature-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.0.0
   [junit4]   2> 17384 INFO  (SUITE-TestFieldLengthFeature-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in standalone mode on port null
   [junit4]   2> 17384 INFO  (SUITE-TestFieldLengthFeature-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 17384 INFO  (SUITE-TestFieldLengthFeature-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-05-22T18:46:02.644Z
   [junit4]   2> 17384 INFO  (SUITE-TestFieldLengthFeature-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/contrib/solr-ltr/test/J0/temp/solr.ltr.feature.TestFieldLengthFeature_2D6C3E0309DABA87-001/tempDir-001/solr.xml
   [junit4]   2> 17461 INFO  (SUITE-TestFieldLengthFeature-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/contrib/solr-ltr/test/J0/temp/solr.ltr.feature.TestFieldLengthFeature_2D6C3E0309DABA87-001/tempDir-003/cores
   [junit4]   2> 17461 INFO  (SUITE-TestFieldLengthFeature-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
   [junit4]   2> 17493 INFO  (coreLoadExecutor-108-thread-1) [    x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 17537 INFO  (coreLoadExecutor-108-thread-1) [    x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=example
   [junit4]   2> 17550 INFO  (coreLoadExecutor-108-thread-1) [    x:collection1] o.a.s.s.IndexSchema Loaded schema example/1.5 with uniqueid field id
   [junit4]   2> 17561 INFO  (coreLoadExecutor-108-thread-1) [    x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from configset /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/contrib/solr-ltr/test/J0/temp/solr.ltr.feature.TestFieldLengthFeature_2D6C3E0309DABA87-001/tempDir-001/collection1, trusted=true
   [junit4]   2> 17562 INFO  (coreLoadExecutor-108-thread-1) [    x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 17562 INFO  (coreLoadExecutor-108-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/contrib/solr-ltr/test/J0/temp/solr.ltr.feature.TestFieldLengthFeature_2D6C3E0309DABA87-001/tempDir-001/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/contrib/solr-ltr/test/J0/temp/solr.ltr.feature.TestFieldLengthFeature_2D6C3E0309DABA87-001/tempDir-003/cores/core/data/]
   [junit4]   2> 17667 WARN  (coreLoadExecutor-108-thread-1) [    x:collection1] o.a.s.c.RequestHandlers no default request handler is registered (either '/select' or 'standard')
   [junit4]   2> 17668 INFO  (coreLoadExecutor-108-thread-1) [    x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 17668 INFO  (coreLoadExecutor-108-thread-1) [    x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 17669 INFO  (coreLoadExecutor-108-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 15000ms; 
   [junit4]   2> 17669 INFO  (coreLoadExecutor-108-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: if uncommited for 1000ms; 
   [junit4]   2> 17670 INFO  (coreLoadExecutor-108-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5e534a89[collection1] main]
   [junit4]   2> 17671 INFO  (coreLoadExecutor-108-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/contrib/solr-ltr/test/J0/temp/solr.ltr.feature.TestFieldLengthFeature_2D6C3E0309DABA87-001/tempDir-001/collection1/conf
   [junit4]   2> 17671 INFO  (coreLoadExecutor-108-thread-1) [    x:collection1] o.a.s.r.RestManager Registered ManagedResource impl org.apache.solr.ltr.store.rest.ManagedFeatureStore for path /schema/feature-store
   [junit4]   2> 17671 WARN  (coreLoadExecutor-108-thread-1) [    x:collection1] o.a.s.r.ManagedResource No stored data found for /schema/feature-store
   [junit4]   2> 17671 INFO  (coreLoadExecutor-108-thread-1) [    x:collection1] o.a.s.l.s.r.ManagedFeatureStore ------ managed feature ~ loading ------
   [junit4]   2> 17671 INFO  (coreLoadExecutor-108-thread-1) [    x:collection1] o.a.s.r.ManagedResource Notified 1 observers of /schema/feature-store
   [junit4]   2> 17671 INFO  (coreLoadExecutor-108-thread-1) [    x:collection1] o.a.s.r.RestManager Registered new managed resource /schema/feature-store
   [junit4]   2> 17671 INFO  (coreLoadExecutor-108-thread-1) [    x:collection1] o.a.s.r.RestManager Registered ManagedResource impl org.apache.solr.ltr.store.rest.ManagedModelStore for path /schema/model-store
   [junit4]   2> 17672 WARN  (coreLoadExecutor-108-thread-1) [    x:collection1] o.a.s.r.ManagedResource No stored data found for /schema/model-store
   [junit4]   2> 17672 INFO  (coreLoadExecutor-108-thread-1) [    x:collection1] o.a.s.l.s.r.ManagedModelStore INIT model store
   [junit4]   2> 17672 INFO  (coreLoadExecutor-108-thread-1) [    x:collection1] o.a.s.l.s.r.ManagedModelStore ------ managed models ~ loading ------
   [junit4]   2> 17672 INFO  (coreLoadExecutor-108-thread-1) [    x:collection1] o.a.s.r.ManagedResource Notified 1 observers of /schema/model-store
   [junit4]   2> 17672 INFO  (coreLoadExecutor-108-thread-1) [    x:collection1] o.a.s.r.RestManager Registered new managed resource /schema/model-store
   [junit4]   2> 17672 INFO  (coreLoadExecutor-108-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 17672 INFO  (coreLoadExecutor-108-thread-1) [    x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1568123139320184832
   [junit4]   2> 17675 INFO  (searcherExecutor-109-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5e534a89[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 17988 INFO  (SUITE-TestFieldLengthFeature-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.SolrJettyTestBase Jetty Assigned Port#55085
   [junit4]   2> 18084 INFO  (qtp511014148-131) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={stream.body=<add><doc><field+name%3D"id">1</field><field+name%3D"title">w1</field><field+name%3D"description">w1</field></doc></add>}{add=[1 (1568123139716546560)]} 0 34
   [junit4]   2> 18102 INFO  (qtp511014148-132) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={stream.body=<add><doc><field+name%3D"id">2</field><field+name%3D"title">w2+2asd+asdd+didid</field><field+name%3D"description">w2+2asd+asdd+didid</field></doc></add>}{add=[2 (1568123139766878208)]} 0 4
   [junit4]   2> 18111 INFO  (qtp511014148-132) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={stream.body=<add><doc><field+name%3D"id">3</field><field+name%3D"title">w3</field><field+name%3D"description">w3</field></doc></add>}{add=[3 (1568123139780509696)]} 0 6
   [junit4]   2> 18116 INFO  (qtp511014148-132) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={stream.body=<add><doc><field+name%3D"id">4</field><field+name%3D"title">w4</field><field+name%3D"description">w4</field></doc></add>}{add=[4 (1568123139784704000)]} 0 1
   [junit4]   2> 18122 INFO  (qtp511014148-132) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={stream.body=<add><doc><field+name%3D"id">5</field><field+name%3D"title">w5</field><field+name%3D"description">w5</field></doc></add>}{add=[5 (1568123139792044032)]} 0 4
   [junit4]   2> 18139 INFO  (qtp511014148-132) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={stream.body=<add><doc><field+name%3D"id">6</field><field+name%3D"title">w1+w2</field><field+name%3D"description">w1+w2</field></doc></add>}{add=[6 (1568123139808821248)]} 0 6
   [junit4]   2> 18143 INFO  (qtp511014148-132) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={stream.body=<add><doc><field+name%3D"id">7</field><field+name%3D"title">w1+w2+w3+w4+w5</field><field+name%3D"description">w1+w2+w3+w4+w5+w8</field></doc></add>}{add=[7 (1568123139813015552)]} 0 1
   [junit4]   2> 18147 INFO  (qtp511014148-132) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={stream.body=<add><doc><field+name%3D"id">8</field><field+name%3D"title">w1+w1+w1+w2+w2+w8</field><field+name%3D"description">w1+w1+w1+w2+w2</field></doc></add>}{add=[8 (1568123139817209856)]} 0 1
   [junit4]   2> 18150 INFO  (qtp511014148-133) [    x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 18150 INFO  (qtp511014148-133) [    x:collection1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@16f2f5a1 commitCommandVersion:0
   [junit4]   2> 18170 INFO  (qtp511014148-133) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@47489f3d[collection1] main]
   [junit4]   2> 18171 INFO  (qtp511014148-133) [    x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 18172 INFO  (searcherExecutor-109-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@47489f3d[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C8)))}
   [junit4]   2> 18172 INFO  (qtp511014148-133) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={stream.body=<commit/>}{commit=} 0 22
   [junit4]   2> 18184 INFO  (TEST-TestFieldLengthFeature.testRanking-seed#[2D6C3E0309DABA87]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testRanking
   [junit4]   2> 18185 INFO  (TEST-TestFieldLengthFeature.testRanking-seed#[2D6C3E0309DABA87]) [    ] o.a.s.l.TestRerankBase loading feauture 
   [junit4]   2> {
   [junit4]   2> "name":"title-length",
   [junit4]   2> "store":"test",
   [junit4]   2> "class":"org.apache.solr.ltr.feature.FieldLengthFeature",
   [junit4]   2> "params":{"field":"title"}
   [junit4]   2> }
   [junit4]   2>  
   [junit4]   2> 18187 INFO  (qtp511014148-134) [    x:collection1] /solr SolrSchemaRestApi: [Restlet] ServerServlet: component class is null
   [junit4]   2> 18189 INFO  (qtp511014148-134) [    x:collection1] /solr SolrSchemaRestApi: [Restlet] Attaching application: org.apache.solr.rest.SolrSchemaRestApi@40ef8457 to URI: /solr/schema
   [junit4]   2> de maig 22, 2017 7:46:03 PM org.restlet.Application start
   [junit4]   2> INFO: Starting org.apache.solr.rest.SolrSchemaRestApi application
   [junit4]   2> 18196 INFO  (qtp511014148-134) [    x:collection1] o.a.s.r.SolrSchemaRestApi createInboundRoot started for /schema
   [junit4]   2> 18199 INFO  (qtp511014148-134) [    x:collection1] o.a.s.r.RestManager Attached managed resource at path: /feature-store
   [junit4]   2> 18199 INFO  (qtp511014148-134) [    x:collection1] o.a.s.r.RestManager Attached managed resource at path: /managed
   [junit4]   2> 18199 INFO  (qtp511014148-134) [    x:collection1] o.a.s.r.RestManager Attached managed resource at path: /model-store
   [junit4]   2> 18199 INFO  (qtp511014148-134) [    x:collection1] o.a.s.r.RestManager Attached 3 ManagedResource endpoints to Restlet router: /schema
   [junit4]   2> 18199 INFO  (qtp511014148-134) [    x:collection1] o.a.s.r.SolrSchemaRestApi createInboundRoot complete for /schema
   [junit4]   2> 18200 INFO  (qtp511014148-134) [    x:collection1] o.a.s.r.RestManager Found ManagedResource [org.apache.solr.ltr.store.rest.ManagedFeatureStore@68ed7ffd] for /schema/feature-store
   [junit4]   2> 18200 INFO  (qtp511014148-134) [    x:collection1] o.a.s.r.ManagedResource Processing update to /schema/feature-store: {name=title-length, store=test, class=org.apache.solr.ltr.feature.FieldLengthFeature, params={field=title}} is a java.util.LinkedHashMap
   [junit4]   2> 18200 INFO  (qtp511014148-134) [    x:collection1] o.a.s.l.s.r.ManagedFeatureStore register feature based on {name=title-length, store=test, class=org.apache.solr.ltr.feature.FieldLengthFeature, params={field=title}}
   [junit4]   2> 18233 INFO  (qtp511014148-134) [    x:collection1] o.a.s.r.ManagedResourceStorage Saved JSON object to path _schema_feature-store.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/contrib/solr-ltr/test/J0/temp/solr.ltr.feature.TestFieldLengthFeature_2D6C3E0309DABA87-001/tempDir-001/collection1/conf
   [junit4]   2> 18233 INFO  (qtp511014148-134) [    x:collection1] o.a.s.r.RestManager [collection1]  webapp=/solr path=/schema/feature-store params={indent=on&wt=json} status=0 QTime=46
   [junit4]   2> de maig 22, 2017 7:46:03 PM org.restlet.engine.log.LogFilter afterHandle
   [junit4]   2> INFO: 2017-05-22	19:46:03	127.0.0.1	-	127.0.0.1	55085	PUT	/solr/schema/feature-store	wt=json&indent=on	200	-	128	44	https://127.0.0.1:55085	Apache-HttpClient/4.4.1 (Java/1.8.0_121)	-
   [junit4]   2> 18238 WARN  (qtp511014148-134) [    x:collection1] o.e.j.s.ServletHandler Error for /solr/collection1/schema/feature-store
   [junit4]   2> java.lang.AssertionError: Attempted close of response output stream.
   [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter$2$1.close(SolrDispatchFilter.java:517)
   [junit4]   2> 	at org.eclipse.jetty.server.Dispatcher.commitResponse(Dispatcher.java:270)
   [junit4]   2> 	at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:202)
   [junit4]   2> 	at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:74)
   [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:383)
   [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:318)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
   [junit4]   2> 	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
   [junit4]   2> 	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
   [junit4]   2> 	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
   [junit4]   2> 	at org.eclipse.jetty.server.Server.handle(Server.java:534)
   [junit4]   2> 	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
   [junit4]   2> 	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
   [junit4]   2> 	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
   [junit4]   2> 	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
   [junit4]   2> 	at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:202)
   [junit4]   2> 	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
   [junit4]   2> 	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
   [junit4]   2> 	at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 18240 INFO  (TEST-TestFieldLengthFeature.testRanking-seed#[2D6C3E0309DABA87]) [    ] o.a.s.l.TestRerankBase loading model 
   [junit4]   2> {
   [junit4]   2> "name":"title-model",
   [junit4]   2> "store":"test",
   [junit4]   2> "class":"org.apache.solr.ltr.model.LinearModel",
   [junit4]   2> "features":[
   [junit4]   2> 	{ "name":"title-length"}
   [junit4]   2> ]
   [junit4]   2> ,
   [junit4]   2> "params":{"weights":{"title-length":1.0}}
   [junit4]   2> }
   [junit4]   2>  
   [junit4]   2> 18247 INFO  (qtp511014148-134) [    x:collection1] o.a.s.r.RestManager Found ManagedResource [ManagedModelStore [store=ModelStore [availableModels=[]], featureStores=org.apache.solr.ltr.store.rest.ManagedFeatureStore@68ed7ffd]] for /schema/model-store
   [junit4]   2> 18247 INFO  (qtp511014148-134) [    x:collection1] o.a.s.r.ManagedResource Processing update to /schema/model-store: {name=title-model, store=test, class=org.apache.solr.ltr.model.LinearModel, features=[{name=title-length}], params={weights={title-length=1.0}}} is a java.util.LinkedHashMap
   [junit4]   2> 18256 INFO  (qtp511014148-134) [    x:collection1] o.a.s.l.s.r.ManagedModelStore adding model title-model
   [junit4]   2> 18257 INFO  (qtp511014148-134) [    x:collection1] o.a.s.r.ManagedResourceStorage Saved JSON object to path _schema_model-store.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/contrib/solr-ltr/test/J0/temp/solr.ltr.feature.TestFieldLengthFeature_2D6C3E0309DABA87-001/tempDir-001/collection1/conf
   [junit4]   2> 18257 INFO  (qtp511014148-134) [    x:collection1] o.a.s.r.RestManager [collection1]  webapp=/solr path=/schema/model-store params={indent=on&wt=json} status=0 QTime=15
   [junit4]   2> de maig 22, 2017 7:46:03 PM org.restlet.engine.log.LogFilter afterHandle
   [junit4]   2> INFO: 2017-05-22	19:46:03	127.0.0.1	-	127.0.0.1	55085	PUT	/solr/schema/model-store	wt=json&indent=on	200	-	176	13	https://127.0.0.1:55085	Apache-HttpClient/4.4.1 (Java/1.8.0_121)	-
   [junit4]   2> 18261 WARN  (qtp511014148-134) [    x:collection1] o.e.j.s.ServletHandler Error for /solr/collection1/schema/model-store
   [junit4]   2> java.lang.AssertionError: Attempted close of response output stream.
   [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter$2$1.close(SolrDispatchFilter.java:517)
   [junit4]   2> 	at org.eclipse.jetty.server.Dispatcher.commitResponse(Dispatcher.java:270)
   [junit4]   2> 	at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:202)
   [junit4]   2> 	at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:74)
   [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:383)
   [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:318)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
   [junit4]   2> 	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
   [junit4]   2> 	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
   [junit4]   2> 	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
   [junit4]   2> 	at org.eclipse.jetty.server.Server.handle(Server.java:534)
   [junit4]   2> 	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
   [junit4]   2> 	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
   [junit4]   2> 	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
   [junit4]   2> 	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
   [junit4]   2> 	at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:202)
   [junit4]   2> 	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
   [junit4]   2> 	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
   [junit4]   2> 	at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 18278 INFO  (qtp511014148-134) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/query params={q=title:w1&indent=on&fl=*,+score&rows=4&wt=json} hits=4 status=0 QTime=1
   [junit4]   2> 18292 INFO  (qtp511014148-134) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/query params={q=title:w1&indent=on&fl=*,+score&rows=4&wt=json} hits=4 status=0 QTime=0
   [junit4]   2> 18295 INFO  (qtp511014148-134) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/query params={q=title:w1&indent=on&fl=*,+score&rows=4&wt=json} hits=4 status=0 QTime=0
   [junit4]   2> 18297 INFO  (qtp511014148-134) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/query params={q=title:w1&indent=on&fl=*,+score&rows=4&wt=json} hits=4 status=0 QTime=0
   [junit4]   2> 18300 INFO  (qtp511014148-134) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/query params={q=title:w1&indent=on&fl=*,+score&rows=4&wt=json} hits=4 status=0 QTime=0
   [junit4]   2> 18304 INFO  (qtp511014148-134) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/query params={q=title:w1&indent=on&fl=*,+score&rows=4&wt=json&rq={!ltr+model%3Dtitle-model+reRankDocs%3D4}} hits=4 status=0 QTime=2
   [junit4]   2> 18314 INFO  (qtp511014148-134) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/query params={q=title:w1&indent=on&fl=*,+score&rows=4&wt=json&rq={!ltr+model%3Dtitle-model+reRankDocs%3D4}} hits=4 status=0 QTime=1
   [junit4]   2> 18317 ERROR (TEST-TestFieldLengthFeature.testRanking-seed#[2D6C3E0309DABA87]) [    ] o.a.s.u.RestTestBase query failed JSON validation. error=mismatch: '8'!='1' @ response/docs/[0]/id
   [junit4]   2>  expected =/response/docs/[0]/id=='8'
   [junit4]   2>  response = {
   [junit4]   2>   "responseHeader":{
   [junit4]   2>     "status":0,
   [junit4]   2>     "QTime":1,
   [junit4]   2>     "params":{
   [junit4]   2>       "q":"title:w1",
   [junit4]   2>       "indent":"on",
   [junit4]   2>       "fl":"*, score",
   [junit4]   2>       "rows":"4",
   [junit4]   2>       "wt":"json",
   [junit4]   2>       "rq":"{!ltr model=title-model reRankDocs=4}"}},
   [junit4]   2>   "response":{"numFound":4,"start":0,"maxScore":2.95147899E18,"docs":[
   [junit4]   2>       {
   [junit4]   2>         "id":"1",
   [junit4]   2>         "title":"w1",
   [junit4]   2>         "description":"w1",
   [junit4]   2>         "_version_":1568123139716546560,
   [junit4]   2>         "score":2.95147899E18},
   [junit4]   2>       {
   [junit4]   2>         "id":"6",
   [junit4]   2>         "title":"w1 w2",
   [junit4]   2>         "description":"w1 w2",
   [junit4]   2>         "_version_":1568123139808821248,
   [junit4]   2>         "score":2.04963825E18},
   [junit4]   2>       {
   [junit4]   2>         "id":"7",
   [junit4]   2>         "title":"w1 w2 w3 w4 w5",
   [junit4]   2>         "description":"w1 w2 w3 w4 w5 w8",
   [junit4]   2>         "_version_":1568123139813015552,
   [junit4]   2>         "score":7.3786975E17},
   [junit4]   2>       {
   [junit4]   2>         "id":"8",
   [junit4]   2>         "title":"w1 w1 w1 w2 w2 w8",
   [junit4]   2>         "description":"w1 w1 w1 w2 w2",
   [junit4]   2>         "_version_":1568123139817209856,
   [junit4]   2>         "score":5.12409561E17}]
   [junit4]   2>   }}
   [junit4]   2> 
   [junit4]   2>  request = /query?q=title%3Aw1&fl=*%2C+score&rows=4&rq=%7B%21ltr+model%3Dtitle-model+reRankDocs%3D4%7D&wt=json&indent=on
   [junit4]   2> 
   [junit4]   2> 18318 INFO  (TEST-TestFieldLengthFeature.testRanking-seed#[2D6C3E0309DABA87]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testRanking
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestFieldLengthFeature -Dtests.method=testRanking -Dtests.seed=2D6C3E0309DABA87 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=ca-ES -Dtests.timezone=Portugal -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.15s J0 | TestFieldLengthFeature.testRanking <<<
   [junit4]    > Throwable #1: java.lang.RuntimeException: mismatch: '8'!='1' @ response/docs/[0]/id
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([2D6C3E0309DABA87:B25A979D91A0A548]:0)
   [junit4]    > 	at org.apache.solr.util.RestTestBase.assertJQ(RestTestBase.java:248)
   [junit4]    > 	at org.apache.solr.util.RestTestBase.assertJQ(RestTestBase.java:192)
   [junit4]    > 	at org.apache.solr.ltr.feature.TestFieldLengthFeature.testRanking(TestFieldLengthFeature.java:117)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 18327 INFO  (TEST-TestFieldLengthFeature.testIfFieldIsEmptyLengthIsZero-seed#[2D6C3E0309DABA87]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testIfFieldIsEmptyLengthIsZero
   [junit4]   2> 18351 INFO  (qtp511014148-134) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={stream.body=<add><doc><field+name%3D"id">43</field><field+name%3D"title">w11</field><field+name%3D"description"></field></doc></add>}{add=[43 (1568123140030070784)]} 0 3
   [junit4]   2> 18354 INFO  (qtp511014148-134) [    x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 18355 INFO  (qtp511014148-134) [    x:collection1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@16f2f5a1 commitCommandVersion:0
   [junit4]   2> 18412 INFO  (qtp511014148-134) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2ca99ab4[collection1] main]
   [junit4]   2> 18414 INFO  (searcherExecutor-109-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2ca99ab4[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C8) Uninverting(_1(7.0.0):C1)))}
   [junit4]   2> 18415 INFO  (qtp511014148-134) [    x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 18415 INFO  (qtp511014148-134) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={stream.body=<commit/>}{commit=} 0 61
   [junit4]   2> 18417 INFO  (TEST-TestFieldLengthFeature.testIfFieldIsEmptyLengthIsZero-seed#[2D6C3E0309DABA87]) [    ] o.a.s.l.TestRerankBase loading feauture 
   [junit4]   2> {
   [junit4]   2> "name":"description-length3",
   [junit4]   2> "store":"test",
   [junit4]   2> "class":"org.apache.solr.ltr.feature.FieldLengthFeature",
   [junit4]   2> "params":{"field":"description"}
   [junit4]   2> }
   [junit4]   2>  
   [junit4]   2> 18426 INFO  (qtp511014148-134) [    x:collection1] o.a.s.r.RestManager Found ManagedResource [org.apache.solr.ltr.store.rest.ManagedFeatureStore@68ed7ffd] for /schema/feature-store
   [junit4]   2> 18430 INFO  (qtp511014148-134) [    x:collection1] o.a.s.r.ManagedResource Processing update to /schema/feature-store: {name=description-length3, store=test, class=org.apache.solr.ltr.feature.FieldLengthFeature, params={field=description}} is a java.util.LinkedHashMap
   [junit4]   2> 18430 INFO  (qtp511014148-134) [    x:collection1] o.a.s.l.s.r.ManagedFeatureStore register feature based on {name=description-length3, store=test, class=org.apache.solr.ltr.feature.FieldLengthFeature, params={field=description}}
   [junit4]   2> 18431 INFO  (qtp511014148-134) [    x:collection1] o.a.s.r.ManagedResourceStorage Saved JSON object to path _schema_feature-store.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/contrib/solr-ltr/test/J0/temp/solr.ltr.feature.TestFieldLengthFeature_2D6C3E0309DABA87-001/tempDir-001/collection1/conf
   [junit4]   2> 18431 INFO  (qtp511014148-134) [    x:collection1] o.a.s.r.RestManager [collection1]  webapp=/solr path=/schema/feature-store params={indent=on&wt=json} status=0 QTime=8
   [junit4]   2> de maig 22, 2017 7:46:03 PM org.restlet.engine.log.LogFilter afterHandle
   [junit4]   2> INFO: 2017-05-22	19:46:03	127.0.0.1	-	127.0.0.1	55085	PUT	/solr/schema/feature-store	wt=json&indent=on	200	-	141	6	https://127.0.0.1:55085	Apache-HttpClient/4.4.1 (Java/1.8.0_121)	-
   [junit4]   2> 18433 WARN  (qtp511014148-134) [    x:collection1] o.e.j.s.ServletHandler Error for /solr/collection1/schema/feature-store
   [junit4]   2> java.lang.AssertionError: Attempted close of response output stream.
   [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter$2$1.close(SolrDispatchFilter.java:517)
   [junit4]   2> 	at org.eclipse.jetty.server.Dispatcher.commitResponse(Dispatcher.java:270)
   [junit4]   2> 	at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:202)
   [junit4]   2> 	at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:74)
   [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:383)
   [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:318)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
   [junit4]   2> 	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
   [junit4]   2> 	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
   [junit4]   2> 	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
   [junit4]   2> 	at org.eclipse.jetty.server.Server.handle(Server.java:534)
   [junit4]   2> 	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
   [junit4]   2> 	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
   [junit4]   2> 	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
   [junit4]   2> 	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
   [junit4]   2> 	at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:202)
   [junit4]   2> 	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
   [junit4]   2> 	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
   [junit4]   2> 	at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 18434 INFO  (TEST-TestFieldLengthFeature.testIfFieldIsEmptyLengthIsZero-seed#[2D6C3E0309DABA87]) [    ] o.a.s.l.TestRerankBase loading model 
   [junit4]   2> {
   [junit4]   2> "name":"description-model3",
   [junit4]   2> "store":"test",
   [junit4]   2> "class":"org.apache.solr.ltr.model.LinearModel",
   [junit4]   2> "features":[
   [junit4]   2> 	{ "name":"description-length3"}
   [junit4]   2> ]
   [junit4]   2> ,
   [junit4]   2> "params":{"weights":{"description-length3":1.0}}
   [junit4]   2> }
   [junit4]   2>  
   [junit4]   2> 18437 INFO  (qtp511014148-134) [    x:collection1] o.a.s.r.RestManager Found ManagedResource [ManagedModelStore [store=ModelStore [availableModels=[title-model]], featureStores=org.apache.solr.ltr.store.rest.ManagedFeatureStore@68ed7ffd]] for /schema/model-store
   [junit4]   2> 18438 INFO  (qtp511014148-134) [    x:collection1] o.a.s.r.ManagedResource Processing update to /schema/model-store: {name=description-model3, store=test, class=org.apache.solr.ltr.model.LinearModel, features=[{name=description-length3}], params={weights={description-length3=1.0}}} is a java.util.LinkedHashMap
   [junit4]   2> 18438 INFO  (qtp511014148-134) [    x:collection1] o.a.s.l.s.r.ManagedModelStore adding model description-model3
   [junit4]   2> 18438 INFO  (qtp511014148-134) [    x:collection1] o.a.s.r.ManagedResourceStorage Saved JSON object to path _schema_model-store.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/contrib/solr-ltr/test/J0/temp/solr.ltr.feature.TestFieldLengthFeature_2D6C3E0309DABA87-001/tempDir-001/collection1/conf
   [junit4]   2> 18439 INFO  (qtp511014148-134) [    x:collection1] o.a.s.r.RestManager [collection1]  webapp=/solr path=/schema/model-store params={indent=on&wt=json} status=0 QTime=2
   [junit4]   2> de maig 22, 2017 7:46:03 PM org.restlet.engine.log.LogFilter afterHandle
   [junit4]   2> INFO: 2017-05-22	19:46:03	127.0.0.1	-	127.0.0.1	55085	PUT	/solr/schema/model-store	wt=json&indent=on	200	-	197	2	https://127.0.0.1:55085	Apache-HttpClient/4.4.1 (Java/1.8.0_121)	-
   [junit4]   2> 18440 WARN  (qtp511014148-134) [    x:collection1] o.e.j.s.ServletHandler Error for /solr/collection1/schema/model-store
   [junit4]   2> java.lang.AssertionError: Attempted close of response output stream.
   [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter$2$1.close(SolrDispatchFilter.java:517)
   [junit4]   2> 	at org.eclipse.jetty.server.Dispatcher.commitResponse(Dispatcher.java:270)
   [junit4]   2> 	at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:202)
   [junit4]   2> 	at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:74)
   [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:383)
   [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:318)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
   [junit4]   2> 	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
   [junit4]   2> 	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
   [junit4]   2> 	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
   [junit4]   2> 	at org.eclipse.jetty.server.Server.handle(Server.java:534)
   [junit4]   2> 	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
   [junit4]   2> 	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
   [junit4]   2> 	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
   [junit4]   2> 	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
   [junit4]   2> 	at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:202)
   [junit4]   2> 	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
   [junit4]   2> 	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
   [junit4]   2> 	at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 18444 INFO  (qtp511014148-134) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/query params={q=title:w11&indent=on&fl=*,+score&rows=4&wt=json&rq={!ltr+model%3Ddescription-model3+reRankDocs%3D8}} hits=1 status=0 QTime=1
   [junit4]   2> 18445 INFO  (TEST-TestFieldLengthFeature.testIfFieldIsEmptyLengthIsZero-seed#[2D6C3E0309DABA87]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testIfFieldIsEmptyLengthIsZero
   [junit4]   2> 18449 INFO  (TEST-TestFieldLengthFeature.testIfFieldIsMissingInDocumentLengthIsZero-seed#[2D6C3E0309DABA87]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testIfFieldIsMissingInDocumentLengthIsZero
   [junit4]   2> 18462 INFO  (qtp511014148-134) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={stream.body=<add><doc><field+name%3D"id">42</field><field+name%3D"title">w10</field></doc></add>}{add=[42 (1568123140145414144)]} 0 5
   [junit4]   2> 18465 INFO  (qtp511014148-135) [    x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 18465 INFO  (qtp511014148-135) [    x:collection1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@16f2f5a1 commitCommandVersion:0
   [junit4]   2> 18471 INFO  (qtp511014148-135) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2128c521[collection1] main]
   [junit4]   2> 18472 INFO  (searcherExecutor-109-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2128c521[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C8) Uninverting(_1(7.0.0):C1) Uninverting(_2(7.0.0):C1)))}
   [junit4]   2> 18472 INFO  (qtp511014148-135) [    x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 18473 INFO  (qtp511014148-135) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={stream.body=<commit/>}{commit=} 0 7
   [junit4]   2> 18474 INFO  (TEST-TestFieldLengthFeature.testIfFieldIsMissingInDocumentLengthIsZero-seed#[2D6C3E0309DABA87]) [    ] o.a.s.l.TestRerankBase loading feauture 
   [junit4]   2> {
   [junit4]   2> "name":"description-length2",
   [junit4]   2> "store":"test",
   [junit4]   2> "class":"org.apache.solr.ltr.feature.FieldLengthFeature",
   [junit4]   2> "params":{"field":"description"}
   [junit4]   2> }
   [junit4]   2>  
   [junit4]   2> 18480 INFO  (qtp511014148-135) [    x:collection1] o.a.s.r.RestManager Found ManagedResource [org.apache.solr.ltr.store.rest.ManagedFeatureStore@68ed7ffd] for /schema/feature-store
   [junit4]   2> 18480 INFO  (qtp511014148-135) [    x:collection1] o.a.s.r.ManagedResource Processing update to /schema/feature-store: {name=description-length2, store=test, class=org.apache.solr.ltr.feature.FieldLengthFeature, params={field=description}} is a java.util.LinkedHashMap
   [junit4]   2> 18480 INFO  (qtp511014148-135) [    x:collection1] o.a.s.l.s.r.ManagedFeatureStore register feature based on {name=description-length2, store=test, class=org.apache.solr.ltr.feature.FieldLengthFeature, params={field=description}}
   [junit4]   2> 18481 INFO  (qtp511014148-135) [    x:collection1] o.a.s.r.ManagedResourceStorage Saved JSON object to path _schema_feature-store.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/contrib/solr-ltr/test/J0/temp/solr.ltr.feature.TestFieldLengthFeature_2D6C3E0309DABA87-001/tempDir-001/collection1/conf
   [junit4]   2> 18481 INFO  (qtp511014148-135) [    x:collection1] o.a.s.r.RestManager [collection1]  webapp=/solr path=/schema/feature-store params={indent=on&wt=json} status=0 QTime=5
   [junit4]   2> de maig 22, 2017 7:46:03 PM org.restlet.engine.log.LogFilter afterHandle
   [junit4]   2> INFO: 2017-05-22	19:46:03	127.0.0.1	-	127.0.0.1	55085	PUT	/solr/schema/feature-store	wt=json&indent=on	200	-	141	2	https://127.0.0.1:55085	Apache-HttpClient/4.4.1 (Java/1.8.0_121)	-
   [junit4]   2> 18486 WARN  (qtp511014148-135) [    x:collection1] o.e.j.s.ServletHandler Error for /solr/collection1/schema/feature-store
   [junit4]   2> java.lang.AssertionError: Attempted close of response output stream.
   [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter$2$1.close(SolrDispatchFilter.java:517)
   [junit4]   2> 	at org.eclipse.jetty.server.Dispatcher.commitResponse(Dispatcher.java:270)
   [junit4]   2> 	at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:202)
   [junit4]   2> 	at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:74)
   [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:383)
   [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:318)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
   [junit4]   2> 	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
   [junit4]   2> 	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
   [junit4]   2> 	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
   [junit4]   2> 	at org.eclipse.jetty.server.Server.handle(Server.java:534)
   [junit4]   2> 	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
   [junit4]   2> 	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
   [junit4]   2> 	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
   [junit4]   2> 	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
   [junit4]   2> 	at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:202)
   [junit4]   2> 	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
   [junit4]   2> 	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
   [junit4]   2> 	at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 18487 INFO  (TEST-TestFieldLengthFeature.testIfFieldIsMissingInDocumentLengthIsZero-seed#[2D6C3E0309DABA87]) [    ] o.a.s.l.TestRerankBase loading model 
   [junit4]   2> {
   [junit4]   2> "name":"description-model2",
   [junit4]   2> "store":"test",
   [junit4]   2> "class":"org.apache.solr.ltr.model.LinearModel",
   [junit4]   2> "features":[
   [junit4]   2> 	{ "name":"description-length2"}
   [junit4]   2> ]
   [junit4]   2> ,
   [junit4]   2> "params":{"weights":{"description-length2":1.0}}
   [junit4]   2> }
   [junit4]   2>  
   [junit4]   2> 18489 INFO  (qtp511014148-135) [    x:collection1] o.a.s.r.RestManager Found ManagedResource [ManagedModelStore [store=ModelStore [availableModels=[title-model, description-model3]], featureStores=org.apache.solr.ltr.store.rest.ManagedFeatureStore@68ed7ffd]] for /schema/model-store
   [junit4]   2> 18490 INFO  (qtp511014148-135) [    x:collection1] o.a.s.r.ManagedResource Processing update to /schema/model-store: {name=description-model2, store=test, class=org.apache.solr.ltr.model.LinearModel, features=[{name=description-length2}], params={weights={description-length2=1.0}}} is a java.util.LinkedHashMap
   [junit4]   2> 18490 INFO  (qtp511014148-135) [    x:collection1] o.a.s.l.s.r.ManagedModelStore adding model description-model2
   [junit4]   2> 18491 INFO  (qtp511014148-135) [    x:collection1] o.a.s.r.ManagedResourceStorage Saved JSON object to path _schema_model-store.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/contrib/solr-ltr/test/J0/temp/solr.ltr.feature.TestFieldLengthFeature_2D6C3E0309DABA87-001/tempDir-001/collection1/conf
   [junit4]   2> 18491 INFO  (qtp511014148-135) [    x:collection1] o.a.s.r.RestManager [collection1]  webapp=/solr path=/schema/model-store params={indent=on&wt=json} status=0 QTime=2
   [junit4]   2> de maig 22, 2017 7:46:03 PM org.restlet.engine.log.LogFilter afterHandle
   [junit4]   2> INFO: 2017-05-22	19:46:03	127.0.0.1	-	127.0.0.1	55085	PUT	/solr/schema/model-store	wt=json&indent=on	200	-	197	2	https://127.0.0.1:55085	Apache-HttpClient/4.4.1 (Java/1.8.0_121)	-
   [junit4]   2> 18494 WARN  (qtp511014148-135) [    x:collection1] o.e.j.s.ServletHandler Error for /solr/collection1/schema/model-store
   [junit4]   2> java.lang.AssertionError: Attempted close of response output stream.
   [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter$2$1.close(SolrDispatchFilter.java:517)
   [junit4]   2> 	at org.eclipse.jetty.server.Dispatcher.commitResponse(Dispatcher.java:270)
   [junit4]   2> 	at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:202)
   [junit4]   2> 	at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:74)
   [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:383)
   [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:318)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
   [junit4]   2> 	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
   [junit4]   2> 	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
   [junit4]   2> 	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
   [junit4]   2> 	at org.eclipse.jetty.server.Server.handle(Server.java:534)
   [junit4]   2> 	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
   [junit4]   2> 	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
   [junit4]   2> 	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
   [junit4]   2> 	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
   [junit4]   2> 	at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:202)
   [junit4]   2> 	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
   [junit4]   2> 	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
   [junit4]   2> 	at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 18498 INFO  (qtp511014148-135) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/query params={q=title:w10&indent=on&fl=*,+score&rows=4&wt=json&rq={!ltr+model%3Ddescription-model2+reRankDocs%3D8}} hits=1 status=0 QTime=1
   [junit4]   2> 18500 INFO  (TEST-TestFieldLengthFeature.testIfFieldIsMissingInDocumentLengthIsZero-seed#[2D6C3E0309DABA87]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testIfFieldIsMissingInDocumentLengthIsZero
   [junit4]   2> 18508 INFO  (SUITE-TestFieldLengthFeature-seed#[2D6C3E0309DABA87]-worker) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@4c4bb677{SSL,[ssl, http/1.1]}{127.0.0.1:0}
   [junit4]   2> 18508 INFO  (SUITE-TestFieldLengthFeature-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=477143633
   [junit4]   2> 18508 INFO  (SUITE-TestFieldLengthFeature-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 18509 INFO  (SUITE-TestFieldLengthFeature-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 18509 INFO  (SUITE-TestFieldLengthFeature-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 18509 INFO  (SUITE-TestFieldLengthFeature-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 18509 INFO  (coreCloseExecutor-114-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@4d78b146
   [junit4]   2> 18519 INFO  (coreCloseExecutor-114-thread-1) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=1299755334
   [junit4]   2> 18532 INFO  (SUITE-TestFieldLengthFeature-seed#[2D6C3E0309DABA87]-worker) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@6e7c7c09{/solr,null,UNAVAILABLE}
   [junit4]   2> 18534 INFO  (SUITE-TestFieldLengthFeature-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> 18534 INFO  (SUITE-TestFieldLengthFeature-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=496085788
   [junit4]   2> 18534 INFO  (SUITE-TestFieldLengthFeature-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 18534 INFO  (SUITE-TestFieldLengthFeature-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 18535 INFO  (SUITE-TestFieldLengthFeature-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 18535 INFO  (SUITE-TestFieldLengthFeature-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 18537 INFO  (coreCloseExecutor-116-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@3c66b393
   [junit4]   2> 18552 INFO  (coreCloseExecutor-116-thread-1) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=1013363603
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/contrib/solr-ltr/test/J0/temp/solr.ltr.feature.TestFieldLengthFeature_2D6C3E0309DABA87-001
   [junit4]   2> NOTE: test params are: codec=DummyCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=DUMMY, chunkSize=10856, maxDocsPerChunk=1, blockSize=3), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=DUMMY, chunkSize=10856, blockSize=3)), sim=RandomSimilarity(queryNorm=true): {}, locale=ca-ES, timezone=Portugal
   [junit4]   2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_121 (64-bit)/cpus=4,threads=1,free=183275864,total=341835776
   [junit4]   2> NOTE: All tests run in this JVM: [TestMultipleAdditiveTreesModel, TestManagedFeatureStore, TestUserTermScorereQDF, TestNoMatchSolrFeature, TestFieldLengthFeature]
   [junit4] Completed [16/33 (1!)] on J0 in 1.74s, 3 tests, 1 error <<< FAILURES!

[...truncated 4 lines...]
   [junit4] Suite: org.apache.solr.ltr.TestSelectiveWeightCreation
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/contrib/solr-ltr/test/J0/temp/solr.ltr.TestSelectiveWeightCreation_2D6C3E0309DABA87-001/init-core-data-001
   [junit4]   2> 18592 WARN  (SUITE-TestSelectiveWeightCreation-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=5 numCloses=5
   [junit4]   2> 18593 INFO  (SUITE-TestSelectiveWeightCreation-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields
   [junit4]   2> 18594 INFO  (SUITE-TestSelectiveWeightCreation-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 18595 INFO  (SUITE-TestSelectiveWeightCreation-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore
   [junit4]   2> 18636 INFO  (SUITE-TestSelectiveWeightCreation-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 18658 INFO  (SUITE-TestSelectiveWeightCreation-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.s.IndexSchema [null] Schema name=example
   [junit4]   2> 18670 INFO  (SUITE-TestSelectiveWeightCreation-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.s.IndexSchema Loaded schema example/1.5 with uniqueid field id
   [junit4]   2> 18803 WARN  (SUITE-TestSelectiveWeightCreation-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.m.r.SolrJmxReporter No JMX server found. Not exposing Solr metrics via JMX.
   [junit4]   2> 18803 WARN  (SUITE-TestSelectiveWeightCreation-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.m.r.SolrJmxReporter No JMX server found. Not exposing Solr metrics via JMX.
   [junit4]   2> 18803 WARN  (SUITE-TestSelectiveWeightCreation-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.m.r.SolrJmxReporter No JMX server found. Not exposing Solr metrics via JMX.
   [junit4]   2> 18859 INFO  (coreLoadExecutor-123-thread-1) [    x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 18875 INFO  (coreLoadExecutor-123-thread-1) [    x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=example
   [junit4]   2> 18892 INFO  (coreLoadExecutor-123-thread-1) [    x:collection1] o.a.s.s.IndexSchema Loaded schema example/1.5 with uniqueid field id
   [junit4]   2> 18896 INFO  (coreLoadExecutor-123-thread-1) [    x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/contrib/ltr/src/test-files/solr/collection1, trusted=true
   [junit4]   2> 18896 WARN  (coreLoadExecutor-123-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter No JMX server found. Not exposing Solr metrics via JMX.
   [junit4]   2> 18897 INFO  (coreLoadExecutor-123-thread-1) [    x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 18897 INFO  (coreLoadExecutor-123-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/contrib/ltr/src/test-files/solr/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/contrib/solr-ltr/test/J0/temp/solr.ltr.TestSelectiveWeightCreation_2D6C3E0309DABA87-001/init-core-data-001/]
   [junit4]   2> 18972 WARN  (coreLoadExecutor-123-thread-1) [    x:collection1] o.a.s.c.RequestHandlers no default request handler is registered (either '/select' or 'standard')
   [junit4]   2> 18983 INFO  (coreLoadExecutor-123-thread-1) [    x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 18983 INFO  (coreLoadExecutor-123-thread-1) [    x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [

[...truncated too long message...]

java:1180)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
   [junit4]   2> 	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
   [junit4]   2> 	at org.eclipse.jetty.server.Server.handle(Server.java:534)
   [junit4]   2> 	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
   [junit4]   2> 	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
   [junit4]   2> 	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
   [junit4]   2> 	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
   [junit4]   2> 	at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 24630 INFO  (qtp1244196891-266) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/query params={q=*:*&indent=on&fl=*,score&rows=4&wt=json&rq={!ltr+reRankDocs%3D10+model%3Dexternalmodel+efi.user_query%3Dw3}} hits=5 status=0 QTime=13
   [junit4]   2> 24631 ERROR (TEST-TestParallelWeightCreation.testLTRScoringQueryParallelWeightCreationResultOrder-seed#[2D6C3E0309DABA87]) [    ] o.a.s.u.RestTestBase query failed JSON validation. error=mismatch: '3'!='4' @ response/docs/[0]/id
   [junit4]   2>  expected =/response/docs/[0]/id=='3'
   [junit4]   2>  response = {
   [junit4]   2>   "responseHeader":{
   [junit4]   2>     "status":0,
   [junit4]   2>     "QTime":13,
   [junit4]   2>     "params":{
   [junit4]   2>       "q":"*:*",
   [junit4]   2>       "indent":"on",
   [junit4]   2>       "fl":"*,score",
   [junit4]   2>       "rows":"4",
   [junit4]   2>       "wt":"json",
   [junit4]   2>       "rq":"{!ltr reRankDocs=10 model=externalmodel efi.user_query=w3}"}},
   [junit4]   2>   "response":{"numFound":5,"start":0,"maxScore":0.36707595,"docs":[
   [junit4]   2>       {
   [junit4]   2>         "id":"4",
   [junit4]   2>         "title":"w3 w3",
   [junit4]   2>         "description":"w4",
   [junit4]   2>         "popularity":4,
   [junit4]   2>         "_version_":1568123146479861760,
   [junit4]   2>         "score":0.36707595},
   [junit4]   2>       {
   [junit4]   2>         "id":"3",
   [junit4]   2>         "title":"w3",
   [junit4]   2>         "description":"w3",
   [junit4]   2>         "popularity":3,
   [junit4]   2>         "_version_":1568123146476716032,
   [junit4]   2>         "score":0.33873552},
   [junit4]   2>       {
   [junit4]   2>         "id":"1",
   [junit4]   2>         "title":"w1 w3",
   [junit4]   2>         "description":"w1",
   [junit4]   2>         "popularity":1,
   [junit4]   2>         "_version_":1568123146437918720,
   [junit4]   2>         "score":0.2545195},
   [junit4]   2>       {
   [junit4]   2>         "id":"2",
   [junit4]   2>         "title":"w2",
   [junit4]   2>         "description":"w2",
   [junit4]   2>         "popularity":2,
   [junit4]   2>         "_version_":1568123146459938816,
   [junit4]   2>         "score":0.0}]
   [junit4]   2>   }}
   [junit4]   2> 
   [junit4]   2>  request = /query?q=*%3A*&fl=*%2Cscore&rows=4&rq=%7B%21ltr+reRankDocs%3D10+model%3Dexternalmodel+efi.user_query%3Dw3%7D&wt=json&indent=on
   [junit4]   2> 
   [junit4]   2> 24631 INFO  (TEST-TestParallelWeightCreation.testLTRScoringQueryParallelWeightCreationResultOrder-seed#[2D6C3E0309DABA87]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testLTRScoringQueryParallelWeightCreationResultOrder
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestParallelWeightCreation -Dtests.method=testLTRScoringQueryParallelWeightCreationResultOrder -Dtests.seed=2D6C3E0309DABA87 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=en-GB -Dtests.timezone=Africa/Cairo -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.98s J0 | TestParallelWeightCreation.testLTRScoringQueryParallelWeightCreationResultOrder <<<
   [junit4]    > Throwable #1: java.lang.RuntimeException: mismatch: '3'!='4' @ response/docs/[0]/id
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([2D6C3E0309DABA87:48201E63C3B8DD6F]:0)
   [junit4]    > 	at org.apache.solr.util.RestTestBase.assertJQ(RestTestBase.java:248)
   [junit4]    > 	at org.apache.solr.util.RestTestBase.assertJQ(RestTestBase.java:192)
   [junit4]    > 	at org.apache.solr.ltr.TestParallelWeightCreation.testLTRScoringQueryParallelWeightCreationResultOrder(TestParallelWeightCreation.java:45)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 24640 INFO  (SUITE-TestParallelWeightCreation-seed#[2D6C3E0309DABA87]-worker) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@28e8927b{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 24640 INFO  (SUITE-TestParallelWeightCreation-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1720679330
   [junit4]   2> 24640 INFO  (SUITE-TestParallelWeightCreation-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 24640 INFO  (SUITE-TestParallelWeightCreation-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 24640 INFO  (SUITE-TestParallelWeightCreation-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 24641 INFO  (SUITE-TestParallelWeightCreation-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 24643 INFO  (coreCloseExecutor-220-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@c7dd208
   [junit4]   2> 24645 INFO  (coreCloseExecutor-220-thread-1) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=209572360
   [junit4]   2> 24652 INFO  (SUITE-TestParallelWeightCreation-seed#[2D6C3E0309DABA87]-worker) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@40eacbb6{/solr,null,UNAVAILABLE}
   [junit4]   2> 24652 INFO  (SUITE-TestParallelWeightCreation-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> 24653 INFO  (SUITE-TestParallelWeightCreation-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1826933119
   [junit4]   2> 24653 INFO  (SUITE-TestParallelWeightCreation-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 24653 INFO  (SUITE-TestParallelWeightCreation-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 24653 INFO  (SUITE-TestParallelWeightCreation-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 24653 INFO  (SUITE-TestParallelWeightCreation-seed#[2D6C3E0309DABA87]-worker) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 24656 INFO  (coreCloseExecutor-222-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@21a1cc80
   [junit4]   2> 24658 INFO  (coreCloseExecutor-222-thread-1) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=564251776
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/contrib/solr-ltr/test/J0/temp/solr.ltr.TestParallelWeightCreation_2D6C3E0309DABA87-001
   [junit4]   2> May 22, 2017 6:46:09 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarity(queryNorm=false): {}, locale=en-GB, timezone=Africa/Cairo
   [junit4]   2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_121 (64-bit)/cpus=4,threads=1,free=205875680,total=340787200
   [junit4]   2> NOTE: All tests run in this JVM: [TestMultipleAdditiveTreesModel, TestManagedFeatureStore, TestUserTermScorereQDF, TestNoMatchSolrFeature, TestFieldLengthFeature, TestSelectiveWeightCreation, TestLTRQParserPlugin, TestLTRWithSort, TestParallelWeightCreation]
   [junit4] Completed [33/33 (4!)] on J0 in 11.06s, 2 tests, 1 error <<< FAILURES!

[...truncated 36939 lines...]

[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1310 - Still unstable

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1310/

1 tests failed.
FAILED:  org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest.test

Error Message:
shard1 is not consistent.  Got 416 from https://127.0.0.1:39631/collection1 (previous client) and got 417 from https://127.0.0.1:38885/collection1

Stack Trace:
java.lang.AssertionError: shard1 is not consistent.  Got 416 from https://127.0.0.1:39631/collection1 (previous client) and got 417 from https://127.0.0.1:38885/collection1
	at __randomizedtesting.SeedInfo.seed([7E477FD0C85CCC0A:F613400A66A0A1F2]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1325)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1304)
	at org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest.test(ChaosMonkeySafeLeaderWithPullReplicasTest.java:209)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11536 lines...]
   [junit4] Suite: org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest_7E477FD0C85CCC0A-001/init-core-data-001
   [junit4]   2> 45715 INFO  (SUITE-ChaosMonkeySafeLeaderWithPullReplicasTest-seed#[7E477FD0C85CCC0A]-worker) [    ] o.a.s.SolrTestCaseJ4 Using TrieFields
   [junit4]   2> 45717 INFO  (SUITE-ChaosMonkeySafeLeaderWithPullReplicasTest-seed#[7E477FD0C85CCC0A]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 45722 INFO  (SUITE-ChaosMonkeySafeLeaderWithPullReplicasTest-seed#[7E477FD0C85CCC0A]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 45726 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.ChaosMonkeySafeLeaderWithPullReplicasTest Starting ChaosMonkey test with 1 shards and 6 nodes
   [junit4]   2> 45736 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 45739 INFO  (Thread-5) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 45739 INFO  (Thread-5) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 45838 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.ZkTestServer start zk server on port:33655
   [junit4]   2> 46108 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running
   [junit4]   2> 46111 ERROR (Thread-5) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 47745 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 47753 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 47756 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 47760 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 47763 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 47766 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 47769 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 47772 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 47782 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 47786 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 47791 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 47830 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
   [junit4]   2> 47977 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest_7E477FD0C85CCC0A-001/control-001/cores/collection1
   [junit4]   2> 48231 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 48274 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1e7c7f3a{/,null,AVAILABLE}
   [junit4]   2> 48334 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@6aafd4e3{SSL,[ssl, http/1.1]}{127.0.0.1:58969}
   [junit4]   2> 48334 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.e.j.s.Server Started @51464ms
   [junit4]   2> 48335 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest_7E477FD0C85CCC0A-001/tempDir-001/control/data, replicaType=NRT, hostContext=/, hostPort=58969, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest_7E477FD0C85CCC0A-001/control-001/cores}
   [junit4]   2> 48355 ERROR (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 48355 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solrâ„¢ version 7.0.0
   [junit4]   2> 48356 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 48356 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 48356 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-05-25T03:02:17.295Z
   [junit4]   2> 48357 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 48370 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 48373 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest_7E477FD0C85CCC0A-001/control-001/solr.xml
   [junit4]   2> 48394 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@7295ef24, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 48423 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33655/solr
   [junit4]   2> 48604 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [n:127.0.0.1:58969_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 48607 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [n:127.0.0.1:58969_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:58969_
   [junit4]   2> 48610 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [n:127.0.0.1:58969_    ] o.a.s.c.Overseer Overseer (id=98020971956862981-127.0.0.1:58969_-n_0000000000) starting
   [junit4]   2> 48829 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [n:127.0.0.1:58969_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58969_
   [junit4]   2> 48840 INFO  (OverseerStateUpdate-98020971956862981-127.0.0.1:58969_-n_0000000000) [n:127.0.0.1:58969_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 49164 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [n:127.0.0.1:58969_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7295ef24
   [junit4]   2> 49165 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [n:127.0.0.1:58969_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7295ef24
   [junit4]   2> 49165 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [n:127.0.0.1:58969_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7295ef24
   [junit4]   2> 49175 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [n:127.0.0.1:58969_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest_7E477FD0C85CCC0A-001/control-001/cores
   [junit4]   2> 49177 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [n:127.0.0.1:58969_    ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
   [junit4]   2> 49197 INFO  (coreLoadExecutor-21-thread-1-processing-n:127.0.0.1:58969_) [n:127.0.0.1:58969_ c:control_collection   x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 49307 INFO  (OverseerStateUpdate-98020971956862981-127.0.0.1:58969_-n_0000000000) [n:127.0.0.1:58969_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 50318 WARN  (coreLoadExecutor-21-thread-1-processing-n:127.0.0.1:58969_) [n:127.0.0.1:58969_ c:control_collection   x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 50319 INFO  (coreLoadExecutor-21-thread-1-processing-n:127.0.0.1:58969_) [n:127.0.0.1:58969_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 50368 INFO  (coreLoadExecutor-21-thread-1-processing-n:127.0.0.1:58969_) [n:127.0.0.1:58969_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 50568 WARN  (coreLoadExecutor-21-thread-1-processing-n:127.0.0.1:58969_) [n:127.0.0.1:58969_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.GeoHashField]. Please consult documentation how to replace it accordingly.
   [junit4]   2> 50576 WARN  (coreLoadExecutor-21-thread-1-processing-n:127.0.0.1:58969_) [n:127.0.0.1:58969_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.LatLonType]. Please consult documentation how to replace it accordingly.
   [junit4]   2> 50595 INFO  (coreLoadExecutor-21-thread-1-processing-n:127.0.0.1:58969_) [n:127.0.0.1:58969_ c:control_collection   x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 50667 INFO  (coreLoadExecutor-21-thread-1-processing-n:127.0.0.1:58969_) [n:127.0.0.1:58969_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection, trusted=true
   [junit4]   2> 50667 INFO  (coreLoadExecutor-21-thread-1-processing-n:127.0.0.1:58969_) [n:127.0.0.1:58969_ c:control_collection   x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.core_node1' (registry 'solr.core.control_collection.shard1.core_node1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7295ef24
   [junit4]   2> 50676 INFO  (coreLoadExecutor-21-thread-1-processing-n:127.0.0.1:58969_) [n:127.0.0.1:58969_ c:control_collection   x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 50676 INFO  (coreLoadExecutor-21-thread-1-processing-n:127.0.0.1:58969_) [n:127.0.0.1:58969_ c:control_collection   x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest_7E477FD0C85CCC0A-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest_7E477FD0C85CCC0A-001/control-001/cores/collection1/data/]
   [junit4]   2> 50685 INFO  (coreLoadExecutor-21-thread-1-processing-n:127.0.0.1:58969_) [n:127.0.0.1:58969_ c:control_collection   x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=18, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=0.2861328125, noCFSRatio=0.6878305367045795]
   [junit4]   2> 50707 WARN  (coreLoadExecutor-21-thread-1-processing-n:127.0.0.1:58969_) [n:127.0.0.1:58969_ c:control_collection   x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 50805 INFO  (coreLoadExecutor-21-thread-1-processing-n:127.0.0.1:58969_) [n:127.0.0.1:58969_ c:control_collection   x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 50805 INFO  (coreLoadExecutor-21-thread-1-processing-n:127.0.0.1:58969_) [n:127.0.0.1:58969_ c:control_collection   x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 50815 INFO  (coreLoadExecutor-21-thread-1-processing-n:127.0.0.1:58969_) [n:127.0.0.1:58969_ c:control_collection   x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 50815 INFO  (coreLoadExecutor-21-thread-1-processing-n:127.0.0.1:58969_) [n:127.0.0.1:58969_ c:control_collection   x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 50819 INFO  (coreLoadExecutor-21-thread-1-processing-n:127.0.0.1:58969_) [n:127.0.0.1:58969_ c:control_collection   x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1541073474, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 50820 INFO  (coreLoadExecutor-21-thread-1-processing-n:127.0.0.1:58969_) [n:127.0.0.1:58969_ c:control_collection   x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4f343be5[collection1] main]
   [junit4]   2> 50825 INFO  (coreLoadExecutor-21-thread-1-processing-n:127.0.0.1:58969_) [n:127.0.0.1:58969_ c:control_collection   x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 50826 INFO  (coreLoadExecutor-21-thread-1-processing-n:127.0.0.1:58969_) [n:127.0.0.1:58969_ c:control_collection   x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 50827 INFO  (coreLoadExecutor-21-thread-1-processing-n:127.0.0.1:58969_) [n:127.0.0.1:58969_ c:control_collection   x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 50830 INFO  (coreLoadExecutor-21-thread-1-processing-n:127.0.0.1:58969_) [n:127.0.0.1:58969_ c:control_collection   x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1568335556529618944
   [junit4]   2> 50835 INFO  (searcherExecutor-22-thread-1-processing-n:127.0.0.1:58969_ x:collection1 c:control_collection) [n:127.0.0.1:58969_ c:control_collection   x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@4f343be5[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 50868 INFO  (coreZkRegister-14-thread-1-processing-n:127.0.0.1:58969_ x:collection1 c:control_collection) [n:127.0.0.1:58969_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 50868 INFO  (coreZkRegister-14-thread-1-processing-n:127.0.0.1:58969_ x:collection1 c:control_collection) [n:127.0.0.1:58969_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 50868 INFO  (coreZkRegister-14-thread-1-processing-n:127.0.0.1:58969_ x:collection1 c:control_collection) [n:127.0.0.1:58969_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:58969/collection1/
   [junit4]   2> 50870 INFO  (coreZkRegister-14-thread-1-processing-n:127.0.0.1:58969_ x:collection1 c:control_collection) [n:127.0.0.1:58969_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 50870 INFO  (coreZkRegister-14-thread-1-processing-n:127.0.0.1:58969_ x:collection1 c:control_collection) [n:127.0.0.1:58969_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy https://127.0.0.1:58969/collection1/ has no replicas
   [junit4]   2> 50870 INFO  (coreZkRegister-14-thread-1-processing-n:127.0.0.1:58969_ x:collection1 c:control_collection) [n:127.0.0.1:58969_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 50896 INFO  (coreZkRegister-14-thread-1-processing-n:127.0.0.1:58969_ x:collection1 c:control_collection) [n:127.0.0.1:58969_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:58969/collection1/ shard1
   [junit4]   2> 51050 INFO  (coreZkRegister-14-thread-1-processing-n:127.0.0.1:58969_ x:collection1 c:control_collection) [n:127.0.0.1:58969_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 51356 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 51358 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33655/solr ready
   [junit4]   2> 51360 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 51489 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest_7E477FD0C85CCC0A-001/shard-1-001/cores/collection1
   [junit4]   2> 51489 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest_7E477FD0C85CCC0A-001/shard-1-001 of type PULL
   [junit4]   2> 51493 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 51500 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@216a0cab{/,null,AVAILABLE}
   [junit4]   2> 51501 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@4d03096d{SSL,[ssl, http/1.1]}{127.0.0.1:58608}
   [junit4]   2> 51501 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.e.j.s.Server Started @54631ms
   [junit4]   2> 51501 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest_7E477FD0C85CCC0A-001/tempDir-001/jetty1, replicaType=PULL, solrconfig=solrconfig.xml, hostContext=/, hostPort=58608, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest_7E477FD0C85CCC0A-001/shard-1-001/cores}
   [junit4]   2> 51502 ERROR (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 51508 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solrâ„¢ version 7.0.0
   [junit4]   2> 51508 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 51508 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 51508 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-05-25T03:02:20.447Z
   [junit4]   2> 51516 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 51516 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest_7E477FD0C85CCC0A-001/shard-1-001/solr.xml
   [junit4]   2> 51524 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@7295ef24, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 51528 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33655/solr
   [junit4]   2> 51556 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [n:127.0.0.1:58608_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 51562 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [n:127.0.0.1:58608_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 51567 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [n:127.0.0.1:58608_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58608_
   [junit4]   2> 51580 INFO  (zkCallback-15-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 51584 INFO  (zkCallback-11-thread-1-processing-n:127.0.0.1:58969_) [n:127.0.0.1:58969_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 51591 INFO  (zkCallback-20-thread-1-processing-n:127.0.0.1:58608_) [n:127.0.0.1:58608_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 51712 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [n:127.0.0.1:58608_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7295ef24
   [junit4]   2> 51726 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [n:127.0.0.1:58608_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7295ef24
   [junit4]   2> 51726 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [n:127.0.0.1:58608_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7295ef24
   [junit4]   2> 51730 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [n:127.0.0.1:58608_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest_7E477FD0C85CCC0A-001/shard-1-001/cores
   [junit4]   2> 51730 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [n:127.0.0.1:58608_    ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
   [junit4]   2> 51736 INFO  (coreLoadExecutor-32-thread-1-processing-n:127.0.0.1:58608_) [n:127.0.0.1:58608_ c:collection1   x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 51741 INFO  (OverseerStateUpdate-98020971956862981-127.0.0.1:58969_-n_0000000000) [n:127.0.0.1:58969_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 52764 WARN  (coreLoadExecutor-32-thread-1-processing-n:127.0.0.1:58608_) [n:127.0.0.1:58608_ c:collection1   x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 52765 INFO  (coreLoadExecutor-32-thread-1-processing-n:127.0.0.1:58608_) [n:127.0.0.1:58608_ c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 52790 INFO  (coreLoadExecutor-32-thread-1-processing-n:127.0.0.1:58608_) [n:127.0.0.1:58608_ c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 52881 INFO  (coreLoadExecutor-32-thread-1-processing-n:127.0.0.1:58608_) [n:127.0.0.1:58608_ c:collection1   x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 52918 INFO  (coreLoadExecutor-32-thread-1-processing-n:127.0.0.1:58608_) [n:127.0.0.1:58608_ c:collection1   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1, trusted=true
   [junit4]   2> 52919 INFO  (coreLoadExecutor-32-thread-1-processing-n:127.0.0.1:58608_) [n:127.0.0.1:58608_ c:collection1   x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.core_node1' (registry 'solr.core.collection1.shard1.core_node1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7295ef24
   [junit4]   2> 52919 INFO  (coreLoadExecutor-32-thread-1-processing-n:127.0.0.1:58608_) [n:127.0.0.1:58608_ c:collection1   x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 52919 INFO  (coreLoadExecutor-32-thread-1-processing-n:127.0.0.1:58608_) [n:127.0.0.1:58608_ c:collection1   x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest_7E477FD0C85CCC0A-001/shard-1-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest_7E477FD0C85CCC0A-001/shard-1-001/cores/collection1/data/]
   [junit4]   2> 52928 INFO  (coreLoadExecutor-32-thread-1-processing-n:127.0.0.1:58608_) [n:127.0.0.1:58608_ c:collection1   x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=18, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=0.2861328125, noCFSRatio=0.6878305367045795]
   [junit4]   2> 52941 WARN  (coreLoadExecutor-32-thread-1-processing-n:127.0.0.1:58608_) [n:127.0.0.1:58608_ c:collection1   x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 53010 INFO  (coreLoadExecutor-32-thread-1-processing-n:127.0.0.1:58608_) [n:127.0.0.1:58608_ c:collection1   x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 53010 INFO  (coreLoadExecutor-32-thread-1-processing-n:127.0.0.1:58608_) [n:127.0.0.1:58608_ c:collection1   x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 53013 INFO  (coreLoadExecutor-32-thread-1-processing-n:127.0.0.1:58608_) [n:127.0.0.1:58608_ c:collection1   x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1541073474, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 53014 INFO  (coreLoadExecutor-32-thread-1-processing-n:127.0.0.1:58608_) [n:127.0.0.1:58608_ c:collection1   x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5656ed9b[collection1] main]
   [junit4]   2> 53016 INFO  (coreLoadExecutor-32-thread-1-processing-n:127.0.0.1:58608_) [n:127.0.0.1:58608_ c:collection1   x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 53018 INFO  (coreLoadExecutor-32-thread-1-processing-n:127.0.0.1:58608_) [n:127.0.0.1:58608_ c:collection1   x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 53019 INFO  (coreLoadExecutor-32-thread-1-processing-n:127.0.0.1:58608_) [n:127.0.0.1:58608_ c:collection1   x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 53022 INFO  (searcherExecutor-33-thread-1-processing-n:127.0.0.1:58608_ x:collection1 c:collection1) [n:127.0.0.1:58608_ c:collection1   x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5656ed9b[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 53027 INFO  (coreZkRegister-27-thread-1-processing-n:127.0.0.1:58608_ x:collection1 c:collection1) [n:127.0.0.1:58608_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController collection1 starting background replication from leader
   [junit4]   2> 53028 INFO  (coreZkRegister-27-thread-1-processing-n:127.0.0.1:58608_ x:collection1 c:collection1) [n:127.0.0.1:58608_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.ReplicateFromLeader Will start replication from leader with poll interval: 00:00:03
   [junit4]   2> 53042 INFO  (coreZkRegister-27-thread-1-processing-n:127.0.0.1:58608_ x:collection1 c:collection1) [n:127.0.0.1:58608_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 3000ms
   [junit4]   2> 53042 WARN  (coreZkRegister-27-thread-1-processing-n:127.0.0.1:58608_ x:collection1 c:collection1) [n:127.0.0.1:58608_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler SolrCloud is enabled for core collection1 but so is old-style replication. Make sure you intend this behavior, it usually indicates a mis-configuration. Master setting is false and slave setting is true
   [junit4]   2> 53411 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest_7E477FD0C85CCC0A-001/shard-2-001/cores/collection1
   [junit4]   2> 53412 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest_7E477FD0C85CCC0A-001/shard-2-001 of type PULL
   [junit4]   2> 53415 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 53420 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6d02f532{/,null,AVAILABLE}
   [junit4]   2> 53426 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@79d421c2{SSL,[ssl, http/1.1]}{127.0.0.1:56918}
   [junit4]   2> 53426 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.e.j.s.Server Started @56556ms
   [junit4]   2> 53426 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest_7E477FD0C85CCC0A-001/tempDir-001/jetty2, replicaType=PULL, solrconfig=solrconfig.xml, hostContext=/, hostPort=56918, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest_7E477FD0C85CCC0A-001/shard-2-001/cores}
   [junit4]   2> 53426 ERROR (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 53426 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solrâ„¢ version 7.0.0
   [junit4]   2> 53426 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 53426 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 53427 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-05-25T03:02:22.366Z
   [junit4]   2> 53436 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 53436 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest_7E477FD0C85CCC0A-001/shard-2-001/solr.xml
   [junit4]   2> 53452 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@7295ef24, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 53457 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33655/solr
   [junit4]   2> 53480 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [n:127.0.0.1:56918_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 53484 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [n:127.0.0.1:56918_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 53487 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [n:127.0.0.1:56918_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56918_
   [junit4]   2> 53491 INFO  (zkCallback-20-thread-1-processing-n:127.0.0.1:58608_) [n:127.0.0.1:58608_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 53491 INFO  (zkCallback-15-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 53492 INFO  (zkCallback-11-thread-1-processing-n:127.0.0.1:58969_) [n:127.0.0.1:58969_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 53495 INFO  (zkCallback-26-thread-1-processing-n:127.0.0.1:56918_) [n:127.0.0.1:56918_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 53612 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [n:127.0.0.1:56918_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7295ef24
   [junit4]   2> 53622 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [n:127.0.0.1:56918_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7295ef24
   [junit4]   2> 53622 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [n:127.0.0.1:56918_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7295ef24
   [junit4]   2> 53625 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [n:127.0.0.1:56918_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest_7E477FD0C85CCC0A-001/shard-2-001/cores
   [junit4]   2> 53626 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [n:127.0.0.1:56918_    ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
   [junit4]   2> 53630 INFO  (coreLoadExecutor-44-thread-1-processing-n:127.0.0.1:56918_) [n:127.0.0.1:56918_ c:collection1   x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 53633 INFO  (OverseerStateUpdate-98020971956862981-127.0.0.1:58969_-n_0000000000) [n:127.0.0.1:58969_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 54660 WARN  (coreLoadExecutor-44-thread-1-processing-n:127.0.0.1:56918_) [n:127.0.0.1:56918_ c:collection1   x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 54661 INFO  (coreLoadExecutor-44-thread-1-processing-n:127.0.0.1:56918_) [n:127.0.0.1:56918_ c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 54682 INFO  (coreLoadExecutor-44-thread-1-processing-n:127.0.0.1:56918_) [n:127.0.0.1:56918_ c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 54781 INFO  (coreLoadExecutor-44-thread-1-processing-n:127.0.0.1:56918_) [n:127.0.0.1:56918_ c:collection1   x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 54820 INFO  (coreLoadExecutor-44-thread-1-processing-n:127.0.0.1:56918_) [n:127.0.0.1:56918_ c:collection1   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1, trusted=true
   [junit4]   2> 54822 INFO  (coreLoadExecutor-44-thread-1-processing-n:127.0.0.1:56918_) [n:127.0.0.1:56918_ c:collection1   x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.core_node2' (registry 'solr.core.collection1.shard1.core_node2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7295ef24
   [junit4]   2> 54822 INFO  (coreLoadExecutor-44-thread-1-processing-n:127.0.0.1:56918_) [n:127.0.0.1:56918_ c:collection1   x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 54822 INFO  (coreLoadExecutor-44-thread-1-processing-n:127.0.0.1:56918_) [n:127.0.0.1:56918_ c:collection1   x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest_7E477FD0C85CCC0A-001/shard-2-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest_7E477FD0C85CCC0A-001/shard-2-001/cores/collection1/data/]
   [junit4]   2> 54826 INFO  (coreLoadExecutor-44-thread-1-processing-n:127.0.0.1:56918_) [n:127.0.0.1:56918_ c:collection1   x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=18, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=0.2861328125, noCFSRatio=0.6878305367045795]
   [junit4]   2> 54847 WARN  (coreLoadExecutor-44-thread-1-processing-n:127.0.0.1:56918_) [n:127.0.0.1:56918_ c:collection1   x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 54913 INFO  (coreLoadExecutor-44-thread-1-processing-n:127.0.0.1:56918_) [n:127.0.0.1:56918_ c:collection1   x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 54913 INFO  (coreLoadExecutor-44-thread-1-processing-n:127.0.0.1:56918_) [n:127.0.0.1:56918_ c:collection1   x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 54915 INFO  (coreLoadExecutor-44-thread-1-processing-n:127.0.0.1:56918_) [n:127.0.0.1:56918_ c:collection1   x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1541073474, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 54916 INFO  (coreLoadExecutor-44-thread-1-processing-n:127.0.0.1:56918_) [n:127.0.0.1:56918_ c:collection1   x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@7b792edf[collection1] main]
   [junit4]   2> 54918 INFO  (coreLoadExecutor-44-thread-1-processing-n:127.0.0.1:56918_) [n:127.0.0.1:56918_ c:collection1   x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 54918 INFO  (coreLoadExecutor-44-thread-1-processing-n:127.0.0.1:56918_) [n:127.0.0.1:56918_ c:collection1   x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 54919 INFO  (coreLoadExecutor-44-thread-1-processing-n:127.0.0.1:56918_) [n:127.0.0.1:56918_ c:collection1   x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 54922 INFO  (searcherExecutor-45-thread-1-processing-n:127.0.0.1:56918_ x:collection1 c:collection1) [n:127.0.0.1:56918_ c:collection1   x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@7b792edf[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 54927 INFO  (coreZkRegister-39-thread-1-processing-n:127.0.0.1:56918_ x:collection1 c:collection1) [n:127.0.0.1:56918_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ZkController collection1 starting background replication from leader
   [junit4]   2> 54927 INFO  (coreZkRegister-39-thread-1-processing-n:127.0.0.1:56918_ x:collection1 c:collection1) [n:127.0.0.1:56918_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ReplicateFromLeader Will start replication from leader with poll interval: 00:00:03
   [junit4]   2> 54929 INFO  (coreZkRegister-39-thread-1-processing-n:127.0.0.1:56918_ x:collection1 c:collection1) [n:127.0.0.1:56918_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 3000ms
   [junit4]   2> 54929 WARN  (coreZkRegister-39-thread-1-processing-n:127.0.0.1:56918_ x:collection1 c:collection1) [n:127.0.0.1:56918_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.h.ReplicationHandler SolrCloud is enabled for core collection1 but so is old-style replication. Make sure you intend this behavior, it usually indicates a mis-configuration. Master setting is false and slave setting is true
   [junit4]   2> 55312 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest_7E477FD0C85CCC0A-001/shard-3-001/cores/collection1
   [junit4]   2> 55312 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest_7E477FD0C85CCC0A-001/shard-3-001 of type PULL
   [junit4]   2> 55315 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 55317 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7c175f39{/,null,AVAILABLE}
   [junit4]   2> 55318 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@41ad1866{SSL,[ssl, http/1.1]}{127.0.0.1:51018}
   [junit4]   2> 55318 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.e.j.s.Server Started @58448ms
   [junit4]   2> 55318 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest_7E477FD0C85CCC0A-001/tempDir-001/jetty3, replicaType=PULL, solrconfig=solrconfig.xml, hostContext=/, hostPort=51018, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest_7E477FD0C85CCC0A-001/shard-3-001/cores}
   [junit4]   2> 55318 ERROR (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 55318 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solrâ„¢ version 7.0.0
   [junit4]   2> 55319 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 55319 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 55319 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-05-25T03:02:24.258Z
   [junit4]   2> 55333 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 55333 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest_7E477FD0C85CCC0A-001/shard-3-001/solr.xml
   [junit4]   2> 55342 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@7295ef24, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 55346 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33655/solr
   [junit4]   2> 55353 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15c3d8dfe07000e, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 55366 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [n:127.0.0.1:51018_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 55370 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [n:127.0.0.1:51018_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 55374 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [n:127.0.0.1:51018_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51018_
   [junit4]   2> 55377 INFO  (zkCallback-26-thread-1-processing-n:127.0.0.1:56918_) [n:127.0.0.1:56918_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 55378 INFO  (zkCallback-32-thread-1-processing-n:127.0.0.1:51018_) [n:127.0.0.1:51018_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 55378 INFO  (zkCallback-11-thread-1-processing-n:127.0.0.1:58969_) [n:127.0.0.1:58969_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 55378 INFO  (zkCallback-15-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 55380 INFO  (zkCallback-20-thread-1-processing-n:127.0.0.1:58608_) [n:127.0.0.1:58608_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 55452 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [n:127.0.0.1:51018_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7295ef24
   [junit4]   2> 55461 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [n:127.0.0.1:51018_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7295ef24
   [junit4]   2> 55461 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [n:127.0.0.1:51018_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7295ef24
   [junit4]   2> 55464 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [n:127.0.0.1:51018_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest_7E477FD0C85CCC0A-001/shard-3-001/cores
   [junit4]   2> 55464 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [n:127.0.0.1:51018_    ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
   [junit4]   2> 55465 INFO  (coreLoadExecutor-56-thread-1-processing-n:127.0.0.1:51018_) [n:127.0.0.1:51018_ c:collection1   x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 55468 INFO  (OverseerStateUpdate-98020971956862981-127.0.0.1:58969_-n_0000000000) [n:127.0.0.1:58969_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 56502 WARN  (coreLoadExecutor-56-thread-1-processing-n:127.0.0.1:51018_) [n:127.0.0.1:51018_ c:collection1   x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 56504 INFO  (coreLoadExecutor-56-thread-1-processing-n:127.0.0.1:51018_) [n:127.0.0.1:51018_ c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 56526 INFO  (coreLoadExecutor-56-thread-1-processing-n:127.0.0.1:51018_) [n:127.0.0.1:51018_ c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 56621 INFO  (coreLoadExecutor-56-thread-1-processing-n:127.0.0.1:51018_) [n:127.0.0.1:51018_ c:collection1   x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 56648 INFO  (coreLoadExecutor-56-thread-1-processing-n:127.0.0.1:51018_) [n:127.0.0.1:51018_ c:collection1   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1, trusted=true
   [junit4]   2> 56649 INFO  (coreLoadExecutor-56-thread-1-processing-n:127.0.0.1:51018_) [n:127.0.0.1:51018_ c:collection1   x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.core_node3' (registry 'solr.core.collection1.shard1.core_node3') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7295ef24
   [junit4]   2> 56649 INFO  (coreLoadExecutor-56-thread-1-processing-n:127.0.0.1:51018_) [n:127.0.0.1:51018_ c:collection1   x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 56649 INFO  (coreLoadExecutor-56-thread-1-processing-n:127.0.0.1:51018_) [n:127.0.0.1:51018_ c:collection1   x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest_7E477FD0C85CCC0A-001/shard-3-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest_7E477FD0C85CCC0A-001/shard-3-001/cores/collection1/data/]
   [junit4]   2> 56656 INFO  (coreLoadExecutor-56-thread-1-processing-n:127.0.0.1:51018_) [n:127.0.0.1:51018_ c:collection1   x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=18, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=0.2861328125, noCFSRatio=0.6878305367045795]
   [junit4]   2> 56667 WARN  (coreLoadExecutor-56-thread-1-processing-n:127.0.0.1:51018_) [n:127.0.0.1:51018_ c:collection1   x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 56733 INFO  (coreLoadExecutor-56-thread-1-processing-n:127.0.0.1:51018_) [n:127.0.0.1:51018_ c:collection1   x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 56733 INFO  (coreLoadExecutor-56-thread-1-processing-n:127.0.0.1:51018_) [n:127.0.0.1:51018_ c:collection1   x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 56735 INFO  (coreLoadExecutor-56-thread-1-processing-n:127.0.0.1:51018_) [n:127.0.0.1:51018_ c:collection1   x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1541073474, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 56736 INFO  (coreLoadExecutor-56-thread-1-processing-n:127.0.0.1:51018_) [n:127.0.0.1:51018_ c:collection1   x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@fcdffb5[collection1] main]
   [junit4]   2> 56738 INFO  (coreLoadExecutor-56-thread-1-processing-n:127.0.0.1:51018_) [n:127.0.0.1:51018_ c:collection1   x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 56738 INFO  (coreLoadExecutor-56-thread-1-processing-n:127.0.0.1:51018_) [n:127.0.0.1:51018_ c:collection1   x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 56739 INFO  (coreLoadExecutor-56-thread-1-processing-n:127.0.0.1:51018_) [n:127.0.0.1:51018_ c:collection1   x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 56743 INFO  (searcherExecutor-57-thread-1-processing-n:127.0.0.1:51018_ x:collection1 c:collection1) [n:127.0.0.1:51018_ c:collection1   x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@fcdffb5[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 56744 INFO  (coreZkRegister-51-thread-1-processing-n:127.0.0.1:51018_ x:collection1 c:collection1) [n:127.0.0.1:51018_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.ZkController collection1 starting background replication from leader
   [junit4]   2> 56744 INFO  (coreZkRegister-51-thread-1-processing-n:127.0.0.1:51018_ x:collection1 c:collection1) [n:127.0.0.1:51018_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.ReplicateFromLeader Will start replication from leader with poll interval: 00:00:03
   [junit4]   2> 56746 INFO  (coreZkRegister-51-thread-1-processing-n:127.0.0.1:51018_ x:collection1 c:collection1) [n:127.0.0.1:51018_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 3000ms
   [junit4]   2> 56746 WARN  (coreZkRegister-51-thread-1-processing-n:127.0.0.1:51018_ x:collection1 c:collection1) [n:127.0.0.1:51018_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.h.ReplicationHandler SolrCloud is enabled for core collection1 but so is old-style replication. Make sure you intend this behavior, it usually indicates a mis-configuration. Master setting is false and slave setting is true
   [junit4]   2> 57141 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest_7E477FD0C85CCC0A-001/shard-4-001/cores/collection1
   [junit4]   2> 57142 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 4 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest_7E477FD0C85CCC0A-001/shard-4-001 of type NRT
   [junit4]   2> 57144 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.e.j.s.Server jetty-9.3.14.v20161028
   [junit4]   2> 57146 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@67452d27{/,null,AVAILABLE}
   [junit4]   2> 57147 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@23c6ea84{SSL,[ssl, http/1.1]}{127.0.0.1:39631}
   [junit4]   2> 57147 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.e.j.s.Server Started @60277ms
   [junit4]   2> 57147 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest_7E477FD0C85CCC0A-001/tempDir-001/jetty4, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/, hostPort=39631, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest_7E477FD0C85CCC0A-001/shard-4-001/cores}
   [junit4]   2> 57147 ERROR (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 57147 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solrâ„¢ version 7.0.0
   [junit4]   2> 57147 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 57147 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 57147 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2017-05-25T03:02:26.086Z
   [junit4]   2> 57156 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 57156 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest_7E477FD0C85CCC0A-001/shard-4-001/solr.xml
   [junit4]   2> 57163 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@7295ef24, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 57167 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33655/solr
   [junit4]   2> 57188 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [n:127.0.0.1:39631_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 57195 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [n:127.0.0.1:39631_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 57199 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [n:127.0.0.1:39631_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39631_
   [junit4]   2> 57202 INFO  (zkCallback-15-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 57202 INFO  (zkCallback-26-thread-1-processing-n:127.0.0.1:56918_) [n:127.0.0.1:56918_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 57205 INFO  (zkCallback-32-thread-1-processing-n:127.0.0.1:51018_) [n:127.0.0.1:51018_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 57205 INFO  (zkCallback-20-thread-1-processing-n:127.0.0.1:58608_) [n:127.0.0.1:58608_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 57205 INFO  (zkCallback-11-thread-1-processing-n:127.0.0.1:58969_) [n:127.0.0.1:58969_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 57205 INFO  (zkCallback-38-thread-1-processing-n:127.0.0.1:39631_) [n:127.0.0.1:39631_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 57307 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [n:127.0.0.1:39631_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7295ef24
   [junit4]   2> 57316 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [n:127.0.0.1:39631_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7295ef24
   [junit4]   2> 57316 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [n:127.0.0.1:39631_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7295ef24
   [junit4]   2> 57319 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [n:127.0.0.1:39631_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest_7E477FD0C85CCC0A-001/shard-4-001/cores
   [junit4]   2> 57319 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [n:127.0.0.1:39631_    ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
   [junit4]   2> 57321 INFO  (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:39631_) [n:127.0.0.1:39631_ c:collection1   x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 57323 INFO  (OverseerStateUpdate-98020971956862981-127.0.0.1:58969_-n_0000000000) [n:127.0.0.1:58969_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 58350 WARN  (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:39631_) [n:127.0.0.1:39631_ c:collection1   x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 58351 INFO  (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:39631_) [n:127.0.0.1:39631_ c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 58372 INFO  (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:39631_) [n:127.0.0.1:39631_ c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 58478 INFO  (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:39631_) [n:127.0.0.1:39631_ c:collection1   x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 58505 ERROR (indexFetcher-38-thread-1) [n:127.0.0.1:58608_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Index fetch failed :org.apache.solr.common.SolrException: No registered leader was found after waiting for 4000ms , collection: collection1 slice: shard1
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:755)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:741)
   [junit4]   2> 	at org.apache.solr.handler.IndexFetcher.getLeaderReplica(IndexFetcher.java:663)
   [junit4]   2> 	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:363)
   [junit4]   2> 	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:328)
   [junit4]   2> 	at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:419)
   [junit4]   2> 	at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$12(ReplicationHandler.java:1183)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   [junit4]   2> 	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
   [junit4]   2> 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
   [junit4]   2> 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 58507 INFO  (indexFetcher-38-thread-1) [n:127.0.0.1:58608_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.h.IndexFetcher Last replication failed, so I'll force replication
   [junit4]   2> 58523 INFO  (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:39631_) [n:127.0.0.1:39631_ c:collection1   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1, trusted=true
   [junit4]   2> 58524 INFO  (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:39631_) [n:127.0.0.1:39631_ c:collection1   x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.core_node4' (registry 'solr.core.collection1.shard1.core_node4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7295ef24
   [junit4]   2> 58524 INFO  (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:39631_) [n:127.0.0.1:39631_ c:collection1   x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 58524 INFO  (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:39631_) [n:127.0.0.1:39631_ c:collection1   x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest_7E477FD0C85CCC0A-001/shard-4-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest_7E477FD0C85CCC0A-001/shard-4-001/cores/collection1/data/]
   [junit4]   2> 58529 INFO  (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:39631_) [n:127.0.0.1:39631_ c:collection1   x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=18, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=0.2861328125, noCFSRatio=0.6878305367045795]
   [junit4]   2> 58539 WARN  (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:39631_) [n:127.0.0.1:39631_ c:collection1   x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 58611 INFO  (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:39631_) [n:127.0.0.1:39631_ c:collection1   x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 58611 INFO  (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:39631_) [n:127.0.0.1:39631_ c:collection1   x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 58614 INFO  (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:39631_) [n:127.0.0.1:39631_ c:collection1   x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 58614 INFO  (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:39631_) [n:127.0.0.1:39631_ c:collection1   x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 58616 INFO  (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:39631_) [n:127.0.0.1:39631_ c:collection1   x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1541073474, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 58617 INFO  (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:39631_) [n:127.0.0.1:39631_ c:collection1   x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1b52addd[collection1] main]
   [junit4]   2> 58619 INFO  (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:39631_) [n:127.0.0.1:39631_ c:collection1   x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 58619 INFO  (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:39631_) [n:127.0.0.1:39631_ c:collection1   x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 58620 INFO  (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:39631_) [n:127.0.0.1:39631_ c:collection1   x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 58626 INFO  (searcherExecutor-69-thread-1-processing-n:127.0.0.1:39631_ x:collection1 c:collection1) [n:127.0.0.1:39631_ c:collection1   x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1b52addd[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 58626 INFO  (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:39631_) [n:127.0.0.1:39631_ c:collection1   x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1568335564704317440
   [junit4]   2> 58638 INFO  (coreZkRegister-63-thread-1-processing-n:127.0.0.1:39631_ x:collection1 c:collection1) [n:127.0.0.1:39631_ c:collection1 s:shard1 r:core_node4 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 58639 INFO  (coreZkRegister-63-thread-1-processing-n:127.0.0.1:39631_ x:collection1 c:collection1) [n:127.0.0.1:39631_ c:collection1 s:shard1 r:core_node4 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 58639 INFO  (coreZkRegister-63-thread-1-processing-n:127.0.0.1:39631_ x:collection1 c:collection1) [n:127.0.0.1:39631_ c:collection1 s:shard1 r:core_node4 x:collection1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:39631/collection1/
   [junit4]   2> 58640 INFO  (coreZkRegister-63-thread-1-processing-n:127.0.0.1:39631_ x:collection1 c:collection1) [n:127.0.0.1:39631_ c:collection1 s:shard1 r:core_node4 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 58640 INFO  (coreZkRegister-63-thread-1-processing-n:127.0.0.1:39631_ x:collection1 c:collection1) [n:127.0.0.1:39631_ c:collection1 s:shard1 r:core_node4 x:collection1] o.a.s.c.SyncStrategy https://127.0.0.1:39631/collection1/ has no replicas
   [junit4]   2> 58640 INFO  (coreZkRegister-63-thread-1-processing-n:127.0.0.1:39631_ x:collection1 c:collection1) [n:127.0.0.1:39631_ c:collection1 s:shard1 r:core_node4 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 58648 INFO  (coreZkRegister-63-thread-1-processing-n:127.0.0.1:39631_ x:collection1 c:collection1) [n:127.0.0.1:39631_ c:collection1 s:shard1 r:core_node4 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:39631/collection1/ shard1
   [junit4]   2> 58758 INFO  (indexFetcher-38-thread-1) [n:127.0.0.1:58608_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.h.IndexFetcher Replica core_node4 is leader but it's state is down, skipping replication
   [junit4]   2> 58791 INFO  (indexFetcher-62-thread-1) [n:127.0.0.1:51018_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.h.IndexFetcher Replica core_node4 is leader but it's state is down, skipping replication
   [junit4]   2> 58799 INFO  (coreZkRegister-51-thread-1-processing-n:127.0.0.1:51018_ x:collection1 c:collection1) [n:127.0.0.1:51018_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.ZkController Core needs to recover:collection1
   [junit4]   2> 58800 INFO  (coreZkRegister-63-thread-1-processing-n:127.0.0.1:39631_ x:collection1 c:collection1) [n:127.0.0.1:39631_ c:collection1 s:shard1 r:core_node4 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 58801 INFO  (indexFetcher-50-thread-1) [n:127.0.0.1:56918_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.h.IndexFetcher Replica core_node4 is leader but it's state is down, skipping replication
   [junit4]   2> 58803 INFO  (updateExecutor-29-thread-1-processing-n:127.0.0.1:51018_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:51018_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.DefaultSolrCoreState Running recovery
   [junit4]   2> 58813 INFO  (recoveryExecutor-30-thread-1-processing-n:127.0.0.1:51018_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:51018_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
   [junit4]   2> 58813 INFO  (recoveryExecutor-30-thread-1-processing-n:127.0.0.1:51018_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:51018_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Publishing state of core [collection1] as recovering, leader is [https://127.0.0.1:39631/collection1/] and I am [https://127.0.0.1:51018/collection1/]
   [junit4]   2> 58815 INFO  (recoveryExecutor-30-thread-1-processing-n:127.0.0.1:51018_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:51018_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Starting Replication Recovery.
   [junit4]   2> 58815 INFO  (recoveryExecutor-30-thread-1-processing-n:127.0.0.1:51018_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:51018_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Stopping background replicate from leader process
   [junit4]   2> 58815 INFO  (recoveryExecutor-30-thread-1-processing-n:127.0.0.1:51018_ x:collection1 s:shard1 c:collec

[...truncated too long message...]

Context I am going to be the leader 127.0.0.1:38885_
   [junit4]   2> 285029 INFO  (zkCallback-238-thread-5-processing-n:127.0.0.1:38885_) [n:127.0.0.1:38885_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 285029 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@23c6ea84{SSL,[ssl, http/1.1]}{127.0.0.1:0}
   [junit4]   2> 285030 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@67452d27{/,null,UNAVAILABLE}
   [junit4]   2> 285031 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.ChaosMonkey monkey: stop jetty! 49516
   [junit4]   2> 285031 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.ChaosMonkey monkey: stop jetty! 38885
   [junit4]   2> 285031 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=129983242
   [junit4]   2> 285031 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 285031 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 285032 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 285032 INFO  (zkCallback-238-thread-4-processing-n:127.0.0.1:38885_) [n:127.0.0.1:38885_ c:collection1 s:shard1 r:core_node6 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 285033 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 286539 WARN  (zkCallback-238-thread-3-processing-n:127.0.0.1:38885_) [n:127.0.0.1:38885_    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 287532 WARN  (zkCallback-238-thread-4-processing-n:127.0.0.1:38885_) [n:127.0.0.1:38885_ c:collection1 s:shard1 r:core_node6 x:collection1] o.a.s.c.SyncStrategy Closed, skipping sync up.
   [junit4]   2> 287533 INFO  (zkCallback-238-thread-4-processing-n:127.0.0.1:38885_) [n:127.0.0.1:38885_ c:collection1 s:shard1 r:core_node6 x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@221ed182
   [junit4]   2> 287537 INFO  (zkCallback-238-thread-4-processing-n:127.0.0.1:38885_) [n:127.0.0.1:38885_ c:collection1 s:shard1 r:core_node6 x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1.shard1.core_node6, tag=572445058
   [junit4]   2> 287542 INFO  (zkCallback-238-thread-4-processing-n:127.0.0.1:38885_) [n:127.0.0.1:38885_ c:collection1 s:shard1 r:core_node6 x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard1.leader, tag=572445058
   [junit4]   2> 287544 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@462ae2ba{SSL,[ssl, http/1.1]}{127.0.0.1:38885}
   [junit4]   2> 287545 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@614c54cc{/,null,UNAVAILABLE}
   [junit4]   2> 287546 ERROR (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 287548 INFO  (TEST-ChaosMonkeySafeLeaderWithPullReplicasTest.test-seed#[7E477FD0C85CCC0A]) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:33655 33655
   [junit4]   2> 287663 INFO  (Thread-5) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:33655 33655
   [junit4]   2> 287666 WARN  (Thread-5) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	40	/solr/aliases.json
   [junit4]   2> 	39	/solr/security.json
   [junit4]   2> 	39	/solr/configs/conf1
   [junit4]   2> 	38	/solr/collections/collection1/state.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	40	/solr/clusterprops.json
   [junit4]   2> 	40	/solr/clusterstate.json
   [junit4]   2> 	8	/solr/overseer_elect/election/98020971956862994-127.0.0.1:39631_-n_0000000004
   [junit4]   2> 	8	/solr/collections/collection1/leader_elect/shard1/election/98020971956862994-core_node4-n_0000000000
   [junit4]   2> 	5	/solr/overseer_elect/election/98020971956863060-127.0.0.1:49516_-n_0000000026
   [junit4]   2> 	4	/solr/overseer_elect/election/98020971956863027-127.0.0.1:58608_-n_0000000015
   [junit4]   2> 	4	/solr/overseer_elect/election/98020971956863006-127.0.0.1:56918_-n_0000000008
   [junit4]   2> 	3	/solr/collections/collection1/leader_elect/shard1/election/98020971956863000-core_node6-n_0000000002
   [junit4]   2> 	3	/solr/overseer_elect/election/98020971956862985-127.0.0.1:58608_-n_0000000001
   [junit4]   2> 	3	/solr/overseer_elect/election/98020971956863072-127.0.0.1:38885_-n_0000000030
   [junit4]   2> 	3	/solr/overseer_elect/election/98020971956863018-127.0.0.1:49516_-n_0000000012
   [junit4]   2> 	2	/solr/overseer_elect/election/98020971956863057-127.0.0.1:51018_-n_0000000025
   [junit4]   2> 	2	/solr/overseer_elect/election/98020971956862981-127.0.0.1:58969_-n_0000000000
   [junit4]   2> 	2	/solr/overseer_elect/election/98020971956863000-127.0.0.1:38885_-n_0000000006
   [junit4]   2> 	2	/solr/overseer_elect/election/98020971956863048-127.0.0.1:56918_-n_0000000022
   [junit4]   2> 	2	/solr/collections/collection1/leader_elect/shard1/election/98020971956863042-core_node6-n_0000000009
   [junit4]   2> 	2	/solr/collections/collection1/leader_elect/shard1/election/98020971956863060-core_node5-n_0000000011
   [junit4]   2> 	2	/solr/collections/collection1/leader_elect/shard1/election/98020971956863018-core_node5-n_0000000005
   [junit4]   2> 	2	/solr/overseer_elect/election/98020971956863036-127.0.0.1:51018_-n_0000000018
   [junit4]   2> 	2	/solr/overseer_elect/election/98020971956863012-127.0.0.1:51018_-n_0000000010
   [junit4]   2> 	2	/solr/overseer_elect/election/98020971956863042-127.0.0.1:38885_-n_0000000020
   [junit4]   2> 	2	/solr/overseer_elect/election/98020971956863051-127.0.0.1:49516_-n_0000000023
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	40	/solr/collections
   [junit4]   2> 	8	/solr/live_nodes
   [junit4]   2> 	2	/solr/overseer/queue
   [junit4]   2> 	2	/solr/overseer/collection-queue-work
   [junit4]   2> 
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=ChaosMonkeySafeLeaderWithPullReplicasTest -Dtests.method=test -Dtests.seed=7E477FD0C85CCC0A -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=nl -Dtests.timezone=WET -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE  242s J1 | ChaosMonkeySafeLeaderWithPullReplicasTest.test <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: shard1 is not consistent.  Got 416 from https://127.0.0.1:39631/collection1 (previous client) and got 417 from https://127.0.0.1:38885/collection1
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([7E477FD0C85CCC0A:F613400A66A0A1F2]:0)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1325)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1304)
   [junit4]    > 	at org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest.test(ChaosMonkeySafeLeaderWithPullReplicasTest.java:209)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest_7E477FD0C85CCC0A-001
   [junit4]   2> May 25, 2017 3:06:16 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70), sim=RandomSimilarity(queryNorm=true): {}, locale=nl, timezone=WET
   [junit4]   2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_121 (64-bit)/cpus=4,threads=1,free=108376120,total=368050176
   [junit4]   2> NOTE: All tests run in this JVM: [TestPointFields, UtilsToolTest, TestLegacyFieldCache, ChaosMonkeySafeLeaderWithPullReplicasTest]
   [junit4] Completed [44/713 (1!)] on J1 in 242.51s, 1 test, 1 failure <<< FAILURES!

[...truncated 45482 lines...]

[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1309 - Failure

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1309/

8 tests failed.
FAILED:  org.apache.lucene.search.join.TestJoinUtil.testEquals_numericJoin

Error Message:
Query shouldn't be equal, because different index readers 

Stack Trace:
java.lang.AssertionError: Query shouldn't be equal, because different index readers 
	at __randomizedtesting.SeedInfo.seed([727888DA3EA9A679:2D1F52677897FF14]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertFalse(Assert.java:68)
	at org.apache.lucene.search.join.TestJoinUtil.testEquals_numericJoin(TestJoinUtil.java:1162)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_dv

Error Message:
java.lang.RuntimeException: Error from server at http://127.0.0.1:49218/solr/test_col: Async exception during distributed update: Error from server at http://127.0.0.1:43671/solr/test_col_shard1_replica_n2: Server Error    request: http://127.0.0.1:43671/solr/test_col_shard1_replica_n2/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A49218%2Fsolr%2Ftest_col_shard5_replica_n1%2F&wt=javabin&version=2 Remote error message: Failed synchronous update on shard StdNode: http://127.0.0.1:55358/solr/test_col_shard1_replica_n1/ update: org.apache.solr.client.solrj.request.UpdateRequest@18d202e6

Stack Trace:
java.util.concurrent.ExecutionException: java.lang.RuntimeException: Error from server at http://127.0.0.1:49218/solr/test_col: Async exception during distributed update: Error from server at http://127.0.0.1:43671/solr/test_col_shard1_replica_n2: Server Error



request: http://127.0.0.1:43671/solr/test_col_shard1_replica_n2/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A49218%2Fsolr%2Ftest_col_shard5_replica_n1%2F&wt=javabin&version=2
Remote error message: Failed synchronous update on shard StdNode: http://127.0.0.1:55358/solr/test_col_shard1_replica_n1/ update: org.apache.solr.client.solrj.request.UpdateRequest@18d202e6
	at __randomizedtesting.SeedInfo.seed([A6FCC6CCFD147F04:90E8A48A77494515]:0)
	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
	at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.checkField(TestStressCloudBlindAtomicUpdates.java:281)
	at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_dv(TestStressCloudBlindAtomicUpdates.java:193)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Error from server at http://127.0.0.1:49218/solr/test_col: Async exception during distributed update: Error from server at http://127.0.0.1:43671/solr/test_col_shard1_replica_n2: Server Error



request: http://127.0.0.1:43671/solr/test_col_shard1_replica_n2/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A49218%2Fsolr%2Ftest_col_shard5_replica_n1%2F&wt=javabin&version=2
Remote error message: Failed synchronous update on shard StdNode: http://127.0.0.1:55358/solr/test_col_shard1_replica_n1/ update: org.apache.solr.client.solrj.request.UpdateRequest@18d202e6
	at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.run(TestStressCloudBlindAtomicUpdates.java:409)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	... 1 more
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:49218/solr/test_col: Async exception during distributed update: Error from server at http://127.0.0.1:43671/solr/test_col_shard1_replica_n2: Server Error



request: http://127.0.0.1:43671/solr/test_col_shard1_replica_n2/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A49218%2Fsolr%2Ftest_col_shard5_replica_n1%2F&wt=javabin&version=2
Remote error message: Failed synchronous update on shard StdNode: http://127.0.0.1:55358/solr/test_col_shard1_replica_n1/ update: org.apache.solr.client.solrj.request.UpdateRequest@18d202e6
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:281)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:270)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:177)
	at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.doRandomAtomicUpdate(TestStressCloudBlindAtomicUpdates.java:368)
	at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.run(TestStressCloudBlindAtomicUpdates.java:397)
	... 6 more


FAILED:  org.apache.solr.core.TestJmxIntegration.testJmxRegistration

Error Message:
org.apache.lucene.store.AlreadyClosedException: this Directory is closed

Stack Trace:
javax.management.RuntimeMBeanException: org.apache.lucene.store.AlreadyClosedException: this Directory is closed
	at __randomizedtesting.SeedInfo.seed([A6FCC6CCFD147F04:282DA2F690552761]:0)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrow(DefaultMBeanServerInterceptor.java:839)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrowMaybeMBeanException(DefaultMBeanServerInterceptor.java:852)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:651)
	at com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678)
	at org.apache.solr.core.TestJmxIntegration.testJmxRegistration(TestJmxIntegration.java:121)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.lucene.store.AlreadyClosedException: this Directory is closed
	at org.apache.lucene.store.BaseDirectory.ensureOpen(BaseDirectory.java:51)
	at org.apache.lucene.store.FSDirectory.listAll(FSDirectory.java:233)
	at org.apache.solr.core.DirectoryFactory.sizeOfDirectory(DirectoryFactory.java:264)
	at org.apache.solr.core.SolrCore.getIndexSize(SolrCore.java:419)
	at org.apache.solr.handler.ReplicationHandler.lambda$initializeMetrics$2(ReplicationHandler.java:867)
	at com.codahale.metrics.JmxReporter$JmxGauge.getValue(JmxReporter.java:188)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:71)
	at sun.reflect.GeneratedMethodAccessor95.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:275)
	at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
	at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
	at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
	at com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:83)
	at com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:206)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:647)
	... 41 more


FAILED:  org.apache.solr.update.HardAutoCommitTest.testCommitWithin

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([A6FCC6CCFD147F04:1C2EA9B47E3A9111]:0)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:898)
	at org.apache.solr.update.HardAutoCommitTest.testCommitWithin(HardAutoCommitTest.java:100)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
	xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
</response>

	request was:q=id:529&qt=standard&start=0&rows=20&version=2.2
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:891)
	... 40 more


FAILED:  org.apache.solr.ltr.TestLTRQParserPlugin.ltrMoreResultsThanReRankedTest

Error Message:
mismatch: '0.09271725'!='0.105360515' @ response/docs/[3]/score

Stack Trace:
java.lang.RuntimeException: mismatch: '0.09271725'!='0.105360515' @ response/docs/[3]/score
	at __randomizedtesting.SeedInfo.seed([86A677DD61788282:4FDEC6BABEB7D543]:0)
	at org.apache.solr.util.RestTestBase.assertJQ(RestTestBase.java:248)
	at org.apache.solr.util.RestTestBase.assertJQ(RestTestBase.java:192)
	at org.apache.solr.ltr.TestLTRQParserPlugin.ltrMoreResultsThanReRankedTest(TestLTRQParserPlugin.java:94)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.ltr.TestParallelWeightCreation.testLTRScoringQueryParallelWeightCreationResultOrder

Error Message:
mismatch: '3'!='4' @ response/docs/[0]/id

Stack Trace:
java.lang.RuntimeException: mismatch: '3'!='4' @ response/docs/[0]/id
	at __randomizedtesting.SeedInfo.seed([86A677DD61788282:E3EA57BDAB1AE56A]:0)
	at org.apache.solr.util.RestTestBase.assertJQ(RestTestBase.java:248)
	at org.apache.solr.util.RestTestBase.assertJQ(RestTestBase.java:192)
	at org.apache.solr.ltr.TestParallelWeightCreation.testLTRScoringQueryParallelWeightCreationResultOrder(TestParallelWeightCreation.java:45)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.ltr.TestSelectiveWeightCreation.testSelectiveWeightsRequestFeaturesFromDifferentStore

Error Message:
mismatch: '3'!='4' @ response/docs/[0]/id

Stack Trace:
java.lang.RuntimeException: mismatch: '3'!='4' @ response/docs/[0]/id
	at __randomizedtesting.SeedInfo.seed([86A677DD61788282:DB976018EC44F5E9]:0)
	at org.apache.solr.util.RestTestBase.assertJQ(RestTestBase.java:248)
	at org.apache.solr.util.RestTestBase.assertJQ(RestTestBase.java:192)
	at org.apache.solr.ltr.TestSelectiveWeightCreation.testSelectiveWeightsRequestFeaturesFromDifferentStore(TestSelectiveWeightCreation.java:230)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.ltr.feature.TestFieldLengthFeature.testRanking

Error Message:
mismatch: '8'!='1' @ response/docs/[0]/id

Stack Trace:
java.lang.RuntimeException: mismatch: '8'!='1' @ response/docs/[0]/id
	at __randomizedtesting.SeedInfo.seed([86A677DD61788282:1990DE43F9029D4D]:0)
	at org.apache.solr.util.RestTestBase.assertJQ(RestTestBase.java:248)
	at org.apache.solr.util.RestTestBase.assertJQ(RestTestBase.java:192)
	at org.apache.solr.ltr.feature.TestFieldLengthFeature.testRanking(TestFieldLengthFeature.java:117)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 7983 lines...]
   [junit4] Suite: org.apache.lucene.search.join.TestJoinUtil
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestJoinUtil -Dtests.method=testEquals_numericJoin -Dtests.seed=727888DA3EA9A679 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=sk -Dtests.timezone=America/Argentina/Ushuaia -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] FAILURE 0.30s J0 | TestJoinUtil.testEquals_numericJoin <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: Query shouldn't be equal, because different index readers 
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([727888DA3EA9A679:2D1F52677897FF14]:0)
   [junit4]    > 	at org.apache.lucene.search.join.TestJoinUtil.testEquals_numericJoin(TestJoinUtil.java:1162)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/build/join/test/J0/temp/lucene.search.join.TestJoinUtil_727888DA3EA9A679-001
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarity(queryNorm=false): {productId=DFR I(ne)Z(0.3), field=DFR GBZ(0.3), price=DFR I(F)3(800.0), subtitle=IB SPL-D3(800.0), name=DFR I(F)LZ(0.3), description=DFR I(F)3(800.0), from=DFR I(n)BZ(0.3), movieId=IB SPL-D2, id=DFR I(F)LZ(0.3), to=DFR I(ne)2, type=DFR I(F)B2, value=IB SPL-L1}, locale=sk, timezone=America/Argentina/Ushuaia
   [junit4]   2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_121 (64-bit)/cpus=4,threads=1,free=264407104,total=351272960
   [junit4]   2> NOTE: All tests run in this JVM: [TestJoinUtil]
   [junit4] Completed [6/8 (1!)] on J0 in 19.14s, 16 tests, 1 failure <<< FAILURES!

[...truncated 4330 lines...]
   [junit4] Suite: org.apache.solr.update.HardAutoCommitTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.update.HardAutoCommitTest_A6FCC6CCFD147F04-001/init-core-data-001
   [junit4]   2> 1439751 WARN  (SUITE-HardAutoCommitTest-seed#[A6FCC6CCFD147F04]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=14 numCloses=14
   [junit4]   2> 1439751 INFO  (SUITE-HardAutoCommitTest-seed#[A6FCC6CCFD147F04]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields
   [junit4]   2> 1439752 INFO  (SUITE-HardAutoCommitTest-seed#[A6FCC6CCFD147F04]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 1439752 INFO  (SUITE-HardAutoCommitTest-seed#[A6FCC6CCFD147F04]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore
   [junit4]   2> 1439753 INFO  (SUITE-HardAutoCommitTest-seed#[A6FCC6CCFD147F04]-worker) [    ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 1439830 INFO  (SUITE-HardAutoCommitTest-seed#[A6FCC6CCFD147F04]-worker) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 1439889 INFO  (SUITE-HardAutoCommitTest-seed#[A6FCC6CCFD147F04]-worker) [    ] o.a.s.s.IndexSchema [null] Schema name=test
   [junit4]   2> 1440108 INFO  (SUITE-HardAutoCommitTest-seed#[A6FCC6CCFD147F04]-worker) [    ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 1440227 INFO  (SUITE-HardAutoCommitTest-seed#[A6FCC6CCFD147F04]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2083e9d
   [junit4]   2> 1440234 INFO  (SUITE-HardAutoCommitTest-seed#[A6FCC6CCFD147F04]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2083e9d
   [junit4]   2> 1440234 INFO  (SUITE-HardAutoCommitTest-seed#[A6FCC6CCFD147F04]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2083e9d
   [junit4]   2> 1440236 INFO  (coreLoadExecutor-3984-thread-1) [    x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 1440277 INFO  (coreLoadExecutor-3984-thread-1) [    x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 1440327 INFO  (coreLoadExecutor-3984-thread-1) [    x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1440603 INFO  (coreLoadExecutor-3984-thread-1) [    x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 1440612 INFO  (coreLoadExecutor-3984-thread-1) [    x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1, trusted=true
   [junit4]   2> 1440612 INFO  (coreLoadExecutor-3984-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2083e9d
   [junit4]   2> 1440612 INFO  (coreLoadExecutor-3984-thread-1) [    x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1440613 INFO  (coreLoadExecutor-3984-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.update.HardAutoCommitTest_A6FCC6CCFD147F04-001/init-core-data-001/]
   [junit4]   2> 1440615 INFO  (coreLoadExecutor-3984-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=39, maxMergeAtOnceExplicit=46, maxMergedSegmentMB=25.44140625, floorSegmentMB=0.8759765625, forceMergeDeletesPctAllowed=7.308900712163352, segmentsPerTier=22.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 1440716 INFO  (coreLoadExecutor-3984-thread-1) [    x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1440716 INFO  (coreLoadExecutor-3984-thread-1) [    x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1440717 INFO  (coreLoadExecutor-3984-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1440717 INFO  (coreLoadExecutor-3984-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1440718 INFO  (coreLoadExecutor-3984-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1300066291, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 1440718 INFO  (coreLoadExecutor-3984-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1ebc7688[collection1] main]
   [junit4]   2> 1440719 INFO  (coreLoadExecutor-3984-thread-1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 1440733 INFO  (coreLoadExecutor-3984-thread-1) [    x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 1440773 INFO  (coreLoadExecutor-3984-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1440773 INFO  (coreLoadExecutor-3984-thread-1) [    x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1568231479097425920
   [junit4]   2> 1440776 INFO  (searcherExecutor-3985-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 1440777 INFO  (searcherExecutor-3985-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 1440777 INFO  (searcherExecutor-3985-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 1440777 INFO  (searcherExecutor-3985-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 1440777 INFO  (SUITE-HardAutoCommitTest-seed#[A6FCC6CCFD147F04]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore end
   [junit4]   2> 1440777 INFO  (searcherExecutor-3985-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 1440778 INFO  (searcherExecutor-3985-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 1440780 INFO  (searcherExecutor-3985-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 1440782 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[A6FCC6CCFD147F04]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testCommitWithin
   [junit4]   2> 1440782 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[A6FCC6CCFD147F04]) [    ] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 1440783 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[A6FCC6CCFD147F04]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@4ebedc09[collection1] realtime]
   [junit4]   2> 1440783 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[A6FCC6CCFD147F04]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={_version_=-9223372036854775807&update.distrib=FROMLEADER&versions=true&wt=json&indent=true}{deleteByQuery=*:* (-9223372036854775807)} 0 1
   [junit4]   2> 1440784 ERROR (TEST-HardAutoCommitTest.testCommitWithin-seed#[A6FCC6CCFD147F04]) [    ] o.a.s.c.CorePropertiesLocator Couldn't load core descriptor from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/core.properties:java.nio.file.NoSuchFileException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/core.properties
   [junit4]   2> 1440788 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[A6FCC6CCFD147F04]) [    ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 1440789 INFO  (searcherExecutor-3985-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 1440789 INFO  (searcherExecutor-3985-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 1440791 INFO  (searcherExecutor-3985-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1ebc7688[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1440904 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[A6FCC6CCFD147F04]) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 1440947 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[A6FCC6CCFD147F04]) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1441198 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[A6FCC6CCFD147F04]) [    ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 1441214 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[A6FCC6CCFD147F04]) [    ] o.a.s.c.CoreContainer Reloading SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1
   [junit4]   2> 1441251 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[A6FCC6CCFD147F04]) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2083e9d
   [junit4]   2> 1441252 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[A6FCC6CCFD147F04]) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.update.HardAutoCommitTest_A6FCC6CCFD147F04-001/init-core-data-001/]
   [junit4]   2> 1441373 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[A6FCC6CCFD147F04]) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1441373 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[A6FCC6CCFD147F04]) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1441375 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[A6FCC6CCFD147F04]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@105c181[collection1] main]
   [junit4]   2> 1441376 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[A6FCC6CCFD147F04]) [    x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 1441377 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[A6FCC6CCFD147F04]) [    x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 1441383 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[A6FCC6CCFD147F04]) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1441383 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[A6FCC6CCFD147F04]) [    x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1568231479737057280
   [junit4]   2> 1441388 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[A6FCC6CCFD147F04]) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@3a4915bd
   [junit4]   2> 1441388 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[A6FCC6CCFD147F04]) [    x:collection1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
   [junit4]   2> 1441389 INFO  (searcherExecutor-3990-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 1441394 INFO  (searcherExecutor-3990-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 1441395 INFO  (searcherExecutor-3990-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 1441395 INFO  (searcherExecutor-3990-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 1441395 INFO  (searcherExecutor-3990-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 1441396 INFO  (searcherExecutor-3990-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 1441396 INFO  (searcherExecutor-3990-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 1441396 INFO  (searcherExecutor-3990-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 1441397 INFO  (searcherExecutor-3990-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 1441400 INFO  (searcherExecutor-3990-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@105c181[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1441400 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[A6FCC6CCFD147F04]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@30bd92b5[collection1] main]
   [junit4]   2> 1441401 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[A6FCC6CCFD147F04]) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@d12371c
   [junit4]   2> 1441401 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[A6FCC6CCFD147F04]) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=219297564
   [junit4]   2> 1441418 INFO  (searcherExecutor-3990-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@30bd92b5[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1441554 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[A6FCC6CCFD147F04]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[529 (1568231479771660288)]} 0 138
   [junit4]   2> 1441554 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[A6FCC6CCFD147F04]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:529&qt=standard&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
   [junit4]   2> 1441558 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[A6FCC6CCFD147F04]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[530 (1568231479919509504)]} 0 142
   [junit4]   2> 1441558 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[A6FCC6CCFD147F04]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:530&qt=standard&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
   [junit4]   2> 1441558 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[A6FCC6CCFD147F04]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {delete=[529 (-1568231479920558080)]} 0 143
   [junit4]   2> 1441559 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[A6FCC6CCFD147F04]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:529&qt=standard&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
   [junit4]   2> 1441559 ERROR (TEST-HardAutoCommitTest.testCommitWithin-seed#[A6FCC6CCFD147F04]) [    x:collection1] o.a.s.SolrTestCaseJ4 REQUEST FAILED: xpath=//result[@numFound=1]
   [junit4]   2> 	xml response was: <?xml version="1.0" encoding="UTF-8"?>
   [junit4]   2> <response>
   [junit4]   2> <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
   [junit4]   2> </response>
   [junit4]   2> 
   [junit4]   2> 	request was:q=id:529&qt=standard&start=0&rows=20&version=2.2
   [junit4]   2> 1441559 ERROR (TEST-HardAutoCommitTest.testCommitWithin-seed#[A6FCC6CCFD147F04]) [    x:collection1] o.a.s.SolrTestCaseJ4 REQUEST FAILED: q=id:529&qt=standard&start=0&rows=20&version=2.2:java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
   [junit4]   2> 	xml response was: <?xml version="1.0" encoding="UTF-8"?>
   [junit4]   2> <response>
   [junit4]   2> <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
   [junit4]   2> </response>
   [junit4]   2> 
   [junit4]   2> 	request was:q=id:529&qt=standard&start=0&rows=20&version=2.2
   [junit4]   2> 	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:891)
   [junit4]   2> 	at org.apache.solr.update.HardAutoCommitTest.testCommitWithin(HardAutoCommitTest.java:100)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:498)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 1441560 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[A6FCC6CCFD147F04]) [    x:collection1] o.a.s.SolrTestCaseJ4 ###Ending testCommitWithin
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=HardAutoCommitTest -Dtests.method=testCommitWithin -Dtests.seed=A6FCC6CCFD147F04 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=hi -Dtests.timezone=Asia/Kabul -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.79s J2 | HardAutoCommitTest.testCommitWithin <<<
   [junit4]    > Throwable #1: java.lang.RuntimeException: Exception during query
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([A6FCC6CCFD147F04:1C2EA9B47E3A9111]:0)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:898)
   [junit4]    > 	at org.apache.solr.update.HardAutoCommitTest.testCommitWithin(HardAutoCommitTest.java:100)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]    > Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
   [junit4]    > 	xml response was: <?xml version="1.0" encoding="UTF-8"?>
   [junit4]    > <response>
   [junit4]    > <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
   [junit4]    > </response>
   [junit4]    > 	request was:q=id:529&qt=standard&start=0&rows=20&version=2.2
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:891)
   [junit4]    > 	... 40 more
   [junit4]   2> 1441569 INFO  (SUITE-HardAutoCommitTest-seed#[A6FCC6CCFD147F04]-worker) [    x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> 1441569 INFO  (SUITE-HardAutoCommitTest-seed#[A6FCC6CCFD147F04]-worker) [    x:collection1] o.a.s.c.CoreContainer Shutting down CoreContainer instance=630252468
   [junit4]   2> 1441569 INFO  (SUITE-HardAutoCommitTest-seed#[A6FCC6CCFD147F04]-worker) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 1441571 INFO  (SUITE-HardAutoCommitTest-seed#[A6FCC6CCFD147F04]-worker) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 1441572 INFO  (SUITE-HardAutoCommitTest-seed#[A6FCC6CCFD147F04]-worker) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 1441577 INFO  (SUITE-HardAutoCommitTest-seed#[A6FCC6CCFD147F04]-worker) [    x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1441589 INFO  (coreCloseExecutor-3994-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@4e85333c
   [junit4]   2> 1441589 INFO  (coreCloseExecutor-3994-thread-1-processing-x:collection1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter close.
   [junit4]   2> 1441590 INFO  (coreCloseExecutor-3994-thread-1-processing-x:collection1) [    x:collection1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@53e951f5 commitCommandVersion:0
   [junit4]   2> 1441609 INFO  (coreCloseExecutor-3994-thread-1-processing-x:collection1) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=1317352252
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.update.HardAutoCommitTest_A6FCC6CCFD147F04-001
   [junit4]   2> May 23, 2017 11:28:04 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 3 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {field_t=PostingsFormat(name=Asserting), multiDefault=FSTOrd50, subject=FSTOrd50, id=FST50, text=PostingsFormat(name=Asserting)}, docValues:{range_facet_l_dv=DocValuesFormat(name=Lucene70), _version_=DocValuesFormat(name=Asserting), intDefault=DocValuesFormat(name=Asserting), range_facet_i_dv=DocValuesFormat(name=Asserting), intDvoDefault=DocValuesFormat(name=Memory), range_facet_l=DocValuesFormat(name=Asserting), timestamp=DocValuesFormat(name=Asserting)}, maxPointsInLeafNode=380, maxMBSortInHeap=5.836481359071707, sim=RandomSimilarity(queryNorm=false): {}, locale=hi, timezone=Asia/Kabul
   [junit4]   2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_121 (64-bit)/cpus=4,threads=1,free=118163904,total=531103744
   [junit4]   2> NOTE: All tests run in this JVM: [TestManagedStopFilterFactory, TermVectorComponentDistributedTest, TestRawTransformer, TestIndexSearcher, TestSQLHandlerNonCloud, TestCloudJSONFacetJoinDomain, TestSimpleQParserPlugin, TestIBSimilarityFactory, AutoCommitTest, DirectUpdateHandlerTest, CurrencyFieldXmlFileTest, TestUseDocValuesAsStored2, JvmMetricsTest, DistributedFacetPivotSmallTest, TestClusterStateMutator, SimpleFacetsTest, BlobRepositoryCloudTest, TestImpersonationWithHadoopAuth, TestSolrQueryParser, ClusterStateTest, HdfsLockFactoryTest, TestCloudManagedSchema, EchoParamsTest, TestNumericTerms32, TestSegmentSorting, MetricUtilsTest, CdcrVersionReplicationTest, RankQueryTest, TestRecoveryHdfs, ExternalFileFieldSortTest, TestExportWriter, RecoveryAfterSoftCommitTest, SuggesterWFSTTest, LeaderInitiatedRecoveryOnCommitTest, TolerantUpdateProcessorTest, TestFieldCacheReopen, TestCollationFieldDocValues, SSLMigrationTest, LukeRequestHandlerTest, TestDistributedMissingSort, LeaderElectionContextKeyTest, ReplicationFactorTest, TestLRUStatsCache, TestTolerantSearch, TestSolrDeletionPolicy1, TestWriterPerf, SegmentsInfoRequestHandlerTest, TestXmlQParserPlugin, BooleanFieldTest, TestCSVResponseWriter, CursorMarkTest, PKIAuthenticationIntegrationTest, SolrSlf4jReporterTest, StatelessScriptUpdateProcessorFactoryTest, TestDynamicFieldCollectionResource, ConfigSetsAPITest, TestJavabinTupleStreamParser, SharedFSAutoReplicaFailoverTest, HdfsBasicDistributedZk2Test, SolrCoreCheckLockOnStartupTest, OverseerStatusTest, TestSolrConfigHandler, TestManagedResource, TestReversedWildcardFilterFactory, ChaosMonkeyNothingIsSafeTest, HdfsUnloadDistributedZkTest, TestLegacyNumericRangeQueryBuilder, TestSort, SubstringBytesRefFilterTest, TestRandomCollapseQParserPlugin, CircularListTest, DateFieldTest, TestFastOutputStream, JsonValidatorTest, TestPayloadCheckQParserPlugin, TestSchemaNameResource, TestAnalyzeInfixSuggestions, TestBinaryField, SolrIndexConfigTest, DocValuesMissingTest, TestCollapseQParserPlugin, BitVectorTest, DeleteStatusTest, SpatialHeatmapFacetsTest, TestRequestForwarding, TlogReplayBufferedWhileIndexingTest, DocValuesMultiTest, SynonymTokenizerTest, TestZkChroot, ActionThrottleTest, TestRandomRequestDistribution, SuggesterTSTTest, TestLocalFSCloudBackupRestore, XmlUpdateRequestHandlerTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, FieldMutatingUpdateProcessorTest, VersionInfoTest, DistributedDebugComponentTest, RecoveryZkTest, TestGroupingSearch, SuggesterTest, AddBlockUpdateTest, HardAutoCommitTest]
   [junit4] Completed [303/713 (1!)] on J2 in 2.15s, 1 test, 1 error <<< FAILURES!

[...truncated 405 lines...]
   [junit4] Suite: org.apache.solr.core.TestJmxIntegration
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.core.TestJmxIntegration_A6FCC6CCFD147F04-001/init-core-data-001
   [junit4]   2> 2511252 WARN  (SUITE-TestJmxIntegration-seed#[A6FCC6CCFD147F04]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
   [junit4]   2> 2511252 INFO  (SUITE-TestJmxIntegration-seed#[A6FCC6CCFD147F04]-worker) [    ] o.a.s.SolrTestCaseJ4 Using TrieFields
   [junit4]   2> 2511254 INFO  (SUITE-TestJmxIntegration-seed#[A6FCC6CCFD147F04]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 2511254 INFO  (SUITE-TestJmxIntegration-seed#[A6FCC6CCFD147F04]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore
   [junit4]   2> 2511254 INFO  (SUITE-TestJmxIntegration-seed#[A6FCC6CCFD147F04]-worker) [    ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 2511283 WARN  (SUITE-TestJmxIntegration-seed#[A6FCC6CCFD147F04]-worker) [    ] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 2511285 INFO  (SUITE-TestJmxIntegration-seed#[A6FCC6CCFD147F04]-worker) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2511309 INFO  (SUITE-TestJmxIntegration-seed#[A6FCC6CCFD147F04]-worker) [    ] o.a.s.s.IndexSchema [null] Schema name=test
   [junit4]   2> 2511418 INFO  (SUITE-TestJmxIntegration-seed#[A6FCC6CCFD147F04]-worker) [    ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 2511458 INFO  (SUITE-TestJmxIntegration-seed#[A6FCC6CCFD147F04]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6aeaa80a
   [junit4]   2> 2511463 INFO  (SUITE-TestJmxIntegration-seed#[A6FCC6CCFD147F04]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6aeaa80a
   [junit4]   2> 2511463 INFO  (SUITE-TestJmxIntegration-seed#[A6FCC6CCFD147F04]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6aeaa80a
   [junit4]   2> 2511465 INFO  (coreLoadExecutor-7112-thread-1) [    x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 2511496 WARN  (coreLoadExecutor-7112-thread-1) [    x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 2511498 INFO  (coreLoadExecutor-7112-thread-1) [    x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2511530 INFO  (coreLoadExecutor-7112-thread-1) [    x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2511630 INFO  (coreLoadExecutor-7112-thread-1) [    x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 2511635 INFO  (coreLoadExecutor-7112-thread-1) [    x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1, trusted=true
   [junit4]   2> 2511636 INFO  (coreLoadExecutor-7112-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6aeaa80a
   [junit4]   2> 2511636 INFO  (coreLoadExecutor-7112-thread-1) [    x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2511636 INFO  (coreLoadExecutor-7112-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.core.TestJmxIntegration_A6FCC6CCFD147F04-001/init-core-data-001/]
   [junit4]   2> 2511637 INFO  (coreLoadExecutor-7112-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=4, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=1.453125, noCFSRatio=0.0]
   [junit4]   2> 2511707 INFO  (coreLoadExecutor-7112-thread-1) [    x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2511707 INFO  (coreLoadExecutor-7112-thread-1) [    x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2511708 INFO  (coreLoadExecutor-7112-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2511708 INFO  (coreLoadExecutor-7112-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2511709 INFO  (coreLoadExecutor-7112-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=49, maxMergeAtOnceExplicit=50, maxMergedSegmentMB=79.5546875, floorSegmentMB=1.84765625, forceMergeDeletesPctAllowed=1.5360643406004981, segmentsPerTier=37.0, maxCFSSegmentSizeMB=0.2236328125, noCFSRatio=1.0
   [junit4]   2> 2511709 INFO  (coreLoadExecutor-7112-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2d63be5a[collection1] main]
   [junit4]   2> 2511710 INFO  (coreLoadExecutor-7112-thread-1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 2511716 INFO  (coreLoadExecutor-7112-thread-1) [    x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 2511740 INFO  (coreLoadExecutor-7112-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2511740 INFO  (searcherExecutor-7113-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 2511740 INFO  (coreLoadExecutor-7112-thread-1) [    x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1568232602161119232
   [junit4]   2> 2511740 INFO  (searcherExecutor-7113-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 2511740 INFO  (searcherExecutor-7113-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 2511740 INFO  (searcherExecutor-7113-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 2511741 INFO  (searcherExecutor-7113-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 2511741 INFO  (searcherExecutor-7113-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 2511741 INFO  (searcherExecutor-7113-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 2511741 INFO  (searcherExecutor-7113-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 2511741 INFO  (searcherExecutor-7113-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 2511743 INFO  (searcherExecutor-7113-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2d63be5a[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2511744 INFO  (SUITE-TestJmxIntegration-seed#[A6FCC6CCFD147F04]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore end
   [junit4]   2> 2511746 INFO  (TEST-TestJmxIntegration.testJmxUpdate-seed#[A6FCC6CCFD147F04]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testJmxUpdate
   [junit4]   2> 2511784 INFO  (TEST-TestJmxIntegration.testJmxUpdate-seed#[A6FCC6CCFD147F04]) [    ] o.a.s.c.TestJmxIntegration Mbeans in server: [solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/admin/luke,name=errors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/admin/logging,name=timeouts, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=QUERY,scope=/dump,name=timeouts, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/admin/threads,name=requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/admin/mbeans,name=clientErrors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/ping,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=QUERY,scope=/dump,name=clientErrors, solr:dom1=core,dom2=collection1,category=UPDATE,scope=update,name=requestTimes, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=QUERY,scope=/dump,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=QUERY,scope=/terms,name=requests, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/properties,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=REPLICATION,scope=/replication,name=replicateAfter, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update,name=handlerStart, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update,name=totalTime, solr:dom1=core,dom2=collection1,category=UPDATE,scope=updateHandler,name=rollbacks, solr:dom1=core,dom2=collection1,category=QUERY,scope=/search-facet-def,name=timeouts, solr:dom1=node,category=ADMIN,scope=/admin/cores,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=UPDATE,scope=updateHandler,name=cumulativeAdds, solr:dom1=node,category=ADMIN,scope=/admin/collections,name=requestTimes, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=REPLICATION,scope=peerSync,name=errors, solr:dom1=core,dom2=collection1,category=HIGHLIGHTER,scope=SolrFragmenter,name=gap.requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=UPDATE,scope=/update/json/docs,name=clientErrors, solr:dom1=node,category=ADMIN,scope=/admin/authorization,name=requestTimes, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/ping,name=errors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/schema,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=TLOG,scope=buffered,name=ops, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/admin/mbeans,name=errors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=QUERY,scope=/stream,name=requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=UPDATE,scope=updateHandler,name=commits, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=QUERY,scope=/stream,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=QUERY,scope=/sql,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=UPDATE,scope=/update/json/docs,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=REPLICATION,scope=peerSync,name=skipped, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=QUERY,scope=/dump,name=timeouts, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=CORE,name=shard, solr:dom1=core,dom2=collection1,category=QUERY,scope=/search-facet-def,name=errors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=QUERY,scope=/get,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=ADMIN,scope=/admin/threads,name=totalTime, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/properties,name=timeouts, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/schema,name=handlerStart, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=UPDATE,scope=update,name=timeouts, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=QUERY,scope=/export,name=clientErrors, solr:dom1=node,category=ADMIN,scope=/admin/authorization,name=handlerStart, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=REPLICATION,scope=/replication,name=isSlave, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update,name=clientErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/search-facet-invariants,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/admin/logging,name=handlerStart, solr:dom1=core,dom2=collection1,category=QUERY,scope=/export,name=totalTime, solr:dom1=core,dom2=collection1,category=QUERY,scope=spellCheckWithWordbreak_Direct,name=errors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=QUERY,scope=/sql,name=timeouts, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=HIGHLIGHTER,scope=SolrBoundaryScanner,name=default.requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/logging,name=requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=QUERY,scope=/debug/dump,name=requestTimes, solr:dom1=node,category=QUERY,scope=httpShardHandler,name=threadPool.httpShardExecutor.completed, solr:dom1=core,dom2=collection1,category=QUERY,scope=/debug/dump,name=clientErrors, java.lang:type=Runtime, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=SEARCHER,name=new, solr:dom1=node,category=ADMIN,scope=/admin/authorization,name=errors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=QUERY,scope=standard,name=handlerStart, solr:dom1=node,category=UPDATE,scope=updateShardHandler,name=threadPool.recoveryExecutor.duration, solr:dom1=core,dom2=collection1,category=CORE,scop

[...truncated too long message...]

=mismatch: '3'!='4' @ response/docs/[0]/id
   [junit4]   2>  expected =/response/docs/[0]/id=='3'
   [junit4]   2>  response = {
   [junit4]   2>   "responseHeader":{
   [junit4]   2>     "status":0,
   [junit4]   2>     "QTime":13,
   [junit4]   2>     "params":{
   [junit4]   2>       "q":"*:*",
   [junit4]   2>       "indent":"on",
   [junit4]   2>       "fl":"*,score",
   [junit4]   2>       "rows":"4",
   [junit4]   2>       "wt":"json",
   [junit4]   2>       "rq":"{!ltr reRankDocs=10 model=externalmodel efi.user_query=w3}"}},
   [junit4]   2>   "response":{"numFound":5,"start":0,"maxScore":0.36707595,"docs":[
   [junit4]   2>       {
   [junit4]   2>         "id":"4",
   [junit4]   2>         "title":"w3 w3",
   [junit4]   2>         "description":"w4",
   [junit4]   2>         "popularity":4,
   [junit4]   2>         "_version_":1568235217740627968,
   [junit4]   2>         "score":0.36707595},
   [junit4]   2>       {
   [junit4]   2>         "id":"3",
   [junit4]   2>         "title":"w3",
   [junit4]   2>         "description":"w3",
   [junit4]   2>         "popularity":3,
   [junit4]   2>         "_version_":1568235217736433664,
   [junit4]   2>         "score":0.33873552},
   [junit4]   2>       {
   [junit4]   2>         "id":"1",
   [junit4]   2>         "title":"w1 w3",
   [junit4]   2>         "description":"w1",
   [junit4]   2>         "popularity":1,
   [junit4]   2>         "_version_":1568235217707073536,
   [junit4]   2>         "score":0.2545195},
   [junit4]   2>       {
   [junit4]   2>         "id":"2",
   [junit4]   2>         "title":"w2",
   [junit4]   2>         "description":"w2",
   [junit4]   2>         "popularity":2,
   [junit4]   2>         "_version_":1568235217726996480,
   [junit4]   2>         "score":0.0}]
   [junit4]   2>   }}
   [junit4]   2> 
   [junit4]   2>  request = /query?q=*%3A*&fl=*%2Cscore&rows=4&rq=%7B%21ltr+reRankDocs%3D10+model%3Dexternalmodel+efi.user_query%3Dw3%7D&wt=json&indent=on
   [junit4]   2> 
   [junit4]   2> 18041 INFO  (TEST-TestParallelWeightCreation.testLTRScoringQueryParallelWeightCreationResultOrder-seed#[86A677DD61788282]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testLTRScoringQueryParallelWeightCreationResultOrder
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestParallelWeightCreation -Dtests.method=testLTRScoringQueryParallelWeightCreationResultOrder -Dtests.seed=86A677DD61788282 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=fr-CA -Dtests.timezone=Australia/NSW -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   1.25s J1 | TestParallelWeightCreation.testLTRScoringQueryParallelWeightCreationResultOrder <<<
   [junit4]    > Throwable #1: java.lang.RuntimeException: mismatch: '3'!='4' @ response/docs/[0]/id
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([86A677DD61788282:E3EA57BDAB1AE56A]:0)
   [junit4]    > 	at org.apache.solr.util.RestTestBase.assertJQ(RestTestBase.java:248)
   [junit4]    > 	at org.apache.solr.util.RestTestBase.assertJQ(RestTestBase.java:192)
   [junit4]    > 	at org.apache.solr.ltr.TestParallelWeightCreation.testLTRScoringQueryParallelWeightCreationResultOrder(TestParallelWeightCreation.java:45)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 18064 INFO  (TEST-TestParallelWeightCreation.testLTRQParserThreadInitialization-seed#[86A677DD61788282]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testLTRQParserThreadInitialization
   [junit4]   2> 18064 INFO  (TEST-TestParallelWeightCreation.testLTRQParserThreadInitialization-seed#[86A677DD61788282]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testLTRQParserThreadInitialization
   [junit4]   2> 18079 INFO  (SUITE-TestParallelWeightCreation-seed#[86A677DD61788282]-worker) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@27966ed6{SSL,[ssl, http/1.1]}{127.0.0.1:0}
   [junit4]   2> 18079 INFO  (SUITE-TestParallelWeightCreation-seed#[86A677DD61788282]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=514573854
   [junit4]   2> 18079 INFO  (SUITE-TestParallelWeightCreation-seed#[86A677DD61788282]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 18080 INFO  (SUITE-TestParallelWeightCreation-seed#[86A677DD61788282]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 18080 INFO  (SUITE-TestParallelWeightCreation-seed#[86A677DD61788282]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 18080 INFO  (SUITE-TestParallelWeightCreation-seed#[86A677DD61788282]-worker) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 18084 INFO  (coreCloseExecutor-104-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@a6d4ee2
   [junit4]   2> 18108 INFO  (coreCloseExecutor-104-thread-1) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=174935778
   [junit4]   2> 18118 INFO  (SUITE-TestParallelWeightCreation-seed#[86A677DD61788282]-worker) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@58b7a51e{/solr,null,UNAVAILABLE}
   [junit4]   2> 18120 INFO  (SUITE-TestParallelWeightCreation-seed#[86A677DD61788282]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> 18120 INFO  (SUITE-TestParallelWeightCreation-seed#[86A677DD61788282]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=101729626
   [junit4]   2> 18120 INFO  (SUITE-TestParallelWeightCreation-seed#[86A677DD61788282]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 18120 INFO  (SUITE-TestParallelWeightCreation-seed#[86A677DD61788282]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 18120 INFO  (SUITE-TestParallelWeightCreation-seed#[86A677DD61788282]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 18120 INFO  (SUITE-TestParallelWeightCreation-seed#[86A677DD61788282]-worker) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 18122 INFO  (coreCloseExecutor-106-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@47eaba05
   [junit4]   2> 18124 INFO  (coreCloseExecutor-106-thread-1) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=1206565381
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/contrib/solr-ltr/test/J1/temp/solr.ltr.TestParallelWeightCreation_86A677DD61788282-001
   [junit4]   2> May 24, 2017 12:27:29 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {_version_=PostingsFormat(name=Asserting), popularity=FSTOrd50, description=PostingsFormat(name=MockRandom), id=FSTOrd50, text=Lucene50(blocksize=128), title=PostingsFormat(name=MockRandom)}, docValues:{}, maxPointsInLeafNode=173, maxMBSortInHeap=7.588669377975336, sim=RandomSimilarity(queryNorm=true): {}, locale=fr-CA, timezone=Australia/NSW
   [junit4]   2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_121 (64-bit)/cpus=4,threads=1,free=301668384,total=345505792
   [junit4]   2> NOTE: All tests run in this JVM: [TestEdisMaxSolrFeature, TestFieldValueFeature, TestLTRScoringQuery, TestUserTermScoreWithQ, TestParallelWeightCreation]
   [junit4] Completed [24/33 (4!)] on J1 in 11.49s, 2 tests, 1 error <<< FAILURES!

[...truncated 28027 lines...]
-documentation-lint:
     [echo] checking for broken html...
    [jtidy] Checking for broken html (such as invalid tags)...
   [delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/build/jtidy_tmp
     [echo] Checking for broken links...
     [exec] 
     [exec] Crawl/parse...
     [exec] 
     [exec] Verify...
     [echo] Checking for missing docs...
     [exec] 
     [exec] build/docs/highlighter/overview-summary.html
     [exec]   missing: org.apache.lucene.search.postingshighlight
     [exec] 
     [exec] build/docs/highlighter/org/apache/lucene/search/postingshighlight/package-summary.html
     [exec]   no package description (missing package.html in src?)
     [exec] 
     [exec] Missing javadocs were found!

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/build.xml:783: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/build.xml:101: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/build.xml:142: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/build.xml:158: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/common-build.xml:2506: exec returned: 1

Total time: 272 minutes 45 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any