You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2017/05/30 23:05:58 UTC
[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1316 - Still
Unstable
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1316/
3 tests failed.
FAILED: org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest.testIntersects { seed=[415F3AEF8B5C61C2:DFD033CD426E2BC0]}
Error Message:
Should have matched I#0:Pt(x=18.0,y=53.0) Q:Pt(x=18.0,y=53.0)
Stack Trace:
java.lang.AssertionError: Should have matched I#0:Pt(x=18.0,y=53.0) Q:Pt(x=18.0,y=53.0)
at __randomizedtesting.SeedInfo.seed([415F3AEF8B5C61C2:DFD033CD426E2BC0]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest.fail(RandomSpatialOpFuzzyPrefixTreeTest.java:399)
at org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest.doTest(RandomSpatialOpFuzzyPrefixTreeTest.java:386)
at org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest.testIntersects(RandomSpatialOpFuzzyPrefixTreeTest.java:138)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_dv
Error Message:
java.lang.RuntimeException: Error from server at http://127.0.0.1:33709/solr/test_col: Async exception during distributed update: Error from server at http://127.0.0.1:52623/solr/test_col_shard2_replica_n1: Server Error request: http://127.0.0.1:52623/solr/test_col_shard2_replica_n1/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A33709%2Fsolr%2Ftest_col_shard1_replica_n2%2F&wt=javabin&version=2 Remote error message: Failed synchronous update on shard StdNode: http://127.0.0.1:35532/solr/test_col_shard2_replica_n2/ update: org.apache.solr.client.solrj.request.UpdateRequest@5a130f41
Stack Trace:
java.util.concurrent.ExecutionException: java.lang.RuntimeException: Error from server at http://127.0.0.1:33709/solr/test_col: Async exception during distributed update: Error from server at http://127.0.0.1:52623/solr/test_col_shard2_replica_n1: Server Error
request: http://127.0.0.1:52623/solr/test_col_shard2_replica_n1/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A33709%2Fsolr%2Ftest_col_shard1_replica_n2%2F&wt=javabin&version=2
Remote error message: Failed synchronous update on shard StdNode: http://127.0.0.1:35532/solr/test_col_shard2_replica_n2/ update: org.apache.solr.client.solrj.request.UpdateRequest@5a130f41
at __randomizedtesting.SeedInfo.seed([B086B20392B0B51A:8692D04518ED8F0B]:0)
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.checkField(TestStressCloudBlindAtomicUpdates.java:281)
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_dv(TestStressCloudBlindAtomicUpdates.java:193)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Error from server at http://127.0.0.1:33709/solr/test_col: Async exception during distributed update: Error from server at http://127.0.0.1:52623/solr/test_col_shard2_replica_n1: Server Error
request: http://127.0.0.1:52623/solr/test_col_shard2_replica_n1/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A33709%2Fsolr%2Ftest_col_shard1_replica_n2%2F&wt=javabin&version=2
Remote error message: Failed synchronous update on shard StdNode: http://127.0.0.1:35532/solr/test_col_shard2_replica_n2/ update: org.apache.solr.client.solrj.request.UpdateRequest@5a130f41
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.run(TestStressCloudBlindAtomicUpdates.java:409)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
... 1 more
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:33709/solr/test_col: Async exception during distributed update: Error from server at http://127.0.0.1:52623/solr/test_col_shard2_replica_n1: Server Error
request: http://127.0.0.1:52623/solr/test_col_shard2_replica_n1/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A33709%2Fsolr%2Ftest_col_shard1_replica_n2%2F&wt=javabin&version=2
Remote error message: Failed synchronous update on shard StdNode: http://127.0.0.1:35532/solr/test_col_shard2_replica_n2/ update: org.apache.solr.client.solrj.request.UpdateRequest@5a130f41
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:610)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:237)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:177)
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.doRandomAtomicUpdate(TestStressCloudBlindAtomicUpdates.java:368)
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.run(TestStressCloudBlindAtomicUpdates.java:397)
... 6 more
FAILED: org.apache.solr.core.TestDynamicLoading.testDynamicLoading
Error Message:
Could not get expected value 'org.apache.solr.core.BlobStoreTestRequestHandler' for path 'overlay/requestHandler/\/test1/class' full output: { "responseHeader":{ "status":0, "QTime":0}, "overlay":{ "znodeVersion":0, "runtimeLib":{"colltest":{ "name":"colltest", "version":1}}}}, from server: null
Stack Trace:
java.lang.AssertionError: Could not get expected value 'org.apache.solr.core.BlobStoreTestRequestHandler' for path 'overlay/requestHandler/\/test1/class' full output: {
"responseHeader":{
"status":0,
"QTime":0},
"overlay":{
"znodeVersion":0,
"runtimeLib":{"colltest":{
"name":"colltest",
"version":1}}}}, from server: null
at __randomizedtesting.SeedInfo.seed([B086B20392B0B51A:68CB9F54656D10BA]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.core.TestSolrConfigHandler.testForResponseElement(TestSolrConfigHandler.java:556)
at org.apache.solr.core.TestDynamicLoading.testDynamicLoading(TestDynamicLoading.java:97)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 9395 lines...]
[junit4] Suite: org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest
[junit4] 2> mai 31, 2017 6:32:21 AM org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(PackedQuadPrefixTree(maxLevels:5,ctx:SpatialContext{geo=false, calculator=CartesianDistCalc, worldBounds=Rect(minX=0.0,maxX=256.0,minY=-128.0,maxY=128.0)},prune:false)))
[junit4] 2> mai 31, 2017 6:32:21 AM org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(GeohashPrefixTree(maxLevels:1,ctx:SpatialContext.GEO)))
[junit4] 2> mai 31, 2017 6:32:21 AM org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(GeohashPrefixTree(maxLevels:1,ctx:SpatialContext.GEO)))
[junit4] 2> mai 31, 2017 6:32:22 AM org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(QuadPrefixTree(maxLevels:8,ctx:SpatialContext{geo=false, calculator=CartesianDistCalc, worldBounds=Rect(minX=0.0,maxX=256.0,minY=-128.0,maxY=128.0)})),pruneLeafyBranches)
[junit4] 2> mai 31, 2017 6:32:22 AM org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(GeohashPrefixTree(maxLevels:3,ctx:SpatialContext.GEO)),pruneLeafyBranches)
[junit4] 2> mai 31, 2017 6:32:22 AM org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(PackedQuadPrefixTree(maxLevels:2,ctx:SpatialContext{geo=false, calculator=CartesianDistCalc, worldBounds=Rect(minX=0.0,maxX=256.0,minY=-128.0,maxY=128.0)},prune:false)),pruneLeafyBranches)
[junit4] 2> mai 31, 2017 6:32:22 AM org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(PackedQuadPrefixTree(maxLevels:8,ctx:SpatialContext{geo=false, calculator=CartesianDistCalc, worldBounds=Rect(minX=0.0,maxX=256.0,minY=-128.0,maxY=128.0)},prune:false)),pruneLeafyBranches)
[junit4] 2> mai 31, 2017 6:32:23 AM org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(GeohashPrefixTree(maxLevels:3,ctx:SpatialContext.GEO)))
[junit4] 2> mai 31, 2017 6:32:23 AM org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(GeohashPrefixTree(maxLevels:1,ctx:SpatialContext.GEO)),pruneLeafyBranches)
[junit4] 2> mai 31, 2017 6:32:23 AM org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(GeohashPrefixTree(maxLevels:1,ctx:SpatialContext.GEO)))
[junit4] 2> mai 31, 2017 6:32:23 AM org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(GeohashPrefixTree(maxLevels:3,ctx:SpatialContext.GEO)),pruneLeafyBranches)
[junit4] 2> mai 31, 2017 6:32:23 AM org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(GeohashPrefixTree(maxLevels:1,ctx:SpatialContext.GEO)),pruneLeafyBranches)
[junit4] 2> mai 31, 2017 6:32:23 AM org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(PackedQuadPrefixTree(maxLevels:8,ctx:SpatialContext{geo=false, calculator=CartesianDistCalc, worldBounds=Rect(minX=0.0,maxX=256.0,minY=-128.0,maxY=128.0)},prune:false)))
[junit4] 2> mai 31, 2017 6:32:24 AM org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(PackedQuadPrefixTree(maxLevels:4,ctx:SpatialContext{geo=false, calculator=CartesianDistCalc, worldBounds=Rect(minX=0.0,maxX=256.0,minY=-128.0,maxY=128.0)},prune:false)))
[junit4] 2> mai 31, 2017 6:32:24 AM org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(GeohashPrefixTree(maxLevels:3,ctx:SpatialContext.GEO)),pointsOnly)
[junit4] 2> mai 31, 2017 6:32:24 AM org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(GeohashPrefixTree(maxLevels:3,ctx:SpatialContext.GEO)))
[junit4] 2> mai 31, 2017 6:32:24 AM org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(GeohashPrefixTree(maxLevels:3,ctx:SpatialContext.GEO)))
[junit4] 2> mai 31, 2017 6:32:24 AM org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(QuadPrefixTree(maxLevels:6,ctx:SpatialContext{geo=false, calculator=CartesianDistCalc, worldBounds=Rect(minX=0.0,maxX=256.0,minY=-128.0,maxY=128.0)})))
[junit4] 2> mai 31, 2017 6:32:24 AM org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(QuadPrefixTree(maxLevels:4,ctx:SpatialContext{geo=false, calculator=CartesianDistCalc, worldBounds=Rect(minX=0.0,maxX=256.0,minY=-128.0,maxY=128.0)})),pruneLeafyBranches)
[junit4] 2> mai 31, 2017 6:32:24 AM org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(GeohashPrefixTree(maxLevels:1,ctx:SpatialContext.GEO)),pruneLeafyBranches)
[junit4] 2> mai 31, 2017 6:32:25 AM org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(GeohashPrefixTree(maxLevels:1,ctx:SpatialContext.GEO)))
[junit4] 2> mai 31, 2017 6:32:25 AM org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(GeohashPrefixTree(maxLevels:1,ctx:SpatialContext.GEO)),pointsOnly)
[junit4] 2> mai 31, 2017 6:32:25 AM org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(QuadPrefixTree(maxLevels:1,ctx:SpatialContext{geo=false, calculator=CartesianDistCalc, worldBounds=Rect(minX=0.0,maxX=256.0,minY=-128.0,maxY=128.0)})))
[junit4] 2> mai 31, 2017 6:32:25 AM org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(GeohashPrefixTree(maxLevels:3,ctx:SpatialContext.GEO)))
[junit4] 2> mai 31, 2017 6:32:25 AM org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(PackedQuadPrefixTree(maxLevels:8,ctx:SpatialContext{geo=false, calculator=CartesianDistCalc, worldBounds=Rect(minX=0.0,maxX=256.0,minY=-128.0,maxY=128.0)},prune:true)),pruneLeafyBranches)
[junit4] 2> mai 31, 2017 6:32:25 AM org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(GeohashPrefixTree(maxLevels:3,ctx:SpatialContext.GEO)),pointsOnly)
[junit4] 2> mai 31, 2017 6:32:25 AM org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(PackedQuadPrefixTree(maxLevels:8,ctx:SpatialContext{geo=false, calculator=CartesianDistCalc, worldBounds=Rect(minX=0.0,maxX=256.0,minY=-128.0,maxY=128.0)},prune:true)),pointsOnly)
[junit4] 2> Ig:Rect(minX=17.0,maxX=19.0,minY=52.0,maxY=54.0) Qg:Rect(minX=17.0,maxX=19.0,minY=52.0,maxY=54.0)
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=RandomSpatialOpFuzzyPrefixTreeTest -Dtests.method=testIntersects -Dtests.seed=415F3AEF8B5C61C2 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=fr-FR -Dtests.timezone=Pacific/Truk -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] FAILURE 0.18s J2 | RandomSpatialOpFuzzyPrefixTreeTest.testIntersects { seed=[415F3AEF8B5C61C2:DFD033CD426E2BC0]} <<<
[junit4] > Throwable #1: java.lang.AssertionError: Should have matched I#0:Pt(x=18.0,y=53.0) Q:Pt(x=18.0,y=53.0)
[junit4] > at __randomizedtesting.SeedInfo.seed([415F3AEF8B5C61C2:DFD033CD426E2BC0]:0)
[junit4] > at org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest.fail(RandomSpatialOpFuzzyPrefixTreeTest.java:399)
[junit4] > at org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest.doTest(RandomSpatialOpFuzzyPrefixTreeTest.java:386)
[junit4] > at org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest.testIntersects(RandomSpatialOpFuzzyPrefixTreeTest.java:138)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> mai 31, 2017 6:32:26 AM org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(GeohashPrefixTree(maxLevels:3,ctx:SpatialContext.GEO)))
[junit4] 2> mai 31, 2017 6:32:26 AM org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(QuadPrefixTree(maxLevels:7,ctx:SpatialContext{geo=false, calculator=CartesianDistCalc, worldBounds=Rect(minX=0.0,maxX=256.0,minY=-128.0,maxY=128.0)})))
[junit4] 2> mai 31, 2017 6:32:26 AM org.apache.lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest setupGrid
[junit4] 2> INFO: Strategy: RecursivePrefixTreeStrategy(SPG:(PackedQuadPrefixTree(maxLevels:8,ctx:SpatialContext{geo=false, calculator=CartesianDistCalc, worldBounds=Rect(minX=0.0,maxX=256.0,minY=-128.0,maxY=128.0)},prune:true)))
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/build/spatial-extras/test/J2/temp/lucene.spatial.prefix.RandomSpatialOpFuzzyPrefixTreeTest_415F3AEF8B5C61C2-001
[junit4] 2> NOTE: test params are: codec=DummyCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=DUMMY, chunkSize=2, maxDocsPerChunk=9, blockSize=213), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=DUMMY, chunkSize=2, blockSize=213)), sim=RandomSimilarity(queryNorm=true): {RandomSpatialOpFuzzyPrefixTreeTest=IB SPL-L3(800.0)}, locale=fr-FR, timezone=Pacific/Truk
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_121 (64-bit)/cpus=4,threads=1,free=252679112,total=339214336
[junit4] 2> NOTE: All tests run in this JVM: [PortedSolr3Test, Geo3dRptTest, RandomSpatialOpFuzzyPrefixTreeTest]
[junit4] Completed [13/23 (1!)] on J2 in 5.04s, 34 tests, 1 failure <<< FAILURES!
[...truncated 3707 lines...]
[junit4] Suite: org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_B086B20392B0B51A-001/init-core-data-001
[junit4] 2> 3184408 INFO (SUITE-TestStressCloudBlindAtomicUpdates-seed#[B086B20392B0B51A]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields
[junit4] 2> 3184410 INFO (SUITE-TestStressCloudBlindAtomicUpdates-seed#[B086B20392B0B51A]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=SSL overhead seems to cause OutOfMemory when stress testing)
[junit4] 2> 3184411 INFO (SUITE-TestStressCloudBlindAtomicUpdates-seed#[B086B20392B0B51A]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 10 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_B086B20392B0B51A-001/tempDir-001
[junit4] 2> 3184411 INFO (SUITE-TestStressCloudBlindAtomicUpdates-seed#[B086B20392B0B51A]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 3184411 INFO (Thread-4684) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 3184411 INFO (Thread-4684) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 3184413 ERROR (Thread-4684) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 3184511 INFO (SUITE-TestStressCloudBlindAtomicUpdates-seed#[B086B20392B0B51A]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:52523
[junit4] 2> 3184528 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15c5b7961010000, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 3184531 INFO (jetty-launcher-3718-thread-5) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 3184532 INFO (jetty-launcher-3718-thread-7) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 3184537 INFO (jetty-launcher-3718-thread-1) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 3184537 INFO (jetty-launcher-3718-thread-10) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 3184537 INFO (jetty-launcher-3718-thread-3) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 3184538 INFO (jetty-launcher-3718-thread-9) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 3184540 INFO (jetty-launcher-3718-thread-6) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 3184541 INFO (jetty-launcher-3718-thread-7) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@47c681f6{/solr,null,AVAILABLE}
[junit4] 2> 3184542 INFO (jetty-launcher-3718-thread-7) [ ] o.e.j.s.AbstractConnector Started ServerConnector@277fe8ff{HTTP/1.1,[http/1.1]}{127.0.0.1:48197}
[junit4] 2> 3184542 INFO (jetty-launcher-3718-thread-7) [ ] o.e.j.s.Server Started @3187948ms
[junit4] 2> 3184542 INFO (jetty-launcher-3718-thread-7) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=48197}
[junit4] 2> 3184542 ERROR (jetty-launcher-3718-thread-7) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3184542 INFO (jetty-launcher-3718-thread-7) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 3184542 INFO (jetty-launcher-3718-thread-7) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3184542 INFO (jetty-launcher-3718-thread-7) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3184543 INFO (jetty-launcher-3718-thread-7) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-05-30T22:28:20.483Z
[junit4] 2> 3184544 INFO (jetty-launcher-3718-thread-8) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 3184545 INFO (jetty-launcher-3718-thread-7) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3184545 INFO (jetty-launcher-3718-thread-2) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 3184547 INFO (jetty-launcher-3718-thread-8) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1f620a04{/solr,null,AVAILABLE}
[junit4] 2> 3184547 INFO (jetty-launcher-3718-thread-8) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3d968316{HTTP/1.1,[http/1.1]}{127.0.0.1:35599}
[junit4] 2> 3184547 INFO (jetty-launcher-3718-thread-8) [ ] o.e.j.s.Server Started @3187953ms
[junit4] 2> 3184547 INFO (jetty-launcher-3718-thread-8) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35599}
[junit4] 2> 3184547 ERROR (jetty-launcher-3718-thread-8) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3184547 INFO (jetty-launcher-3718-thread-8) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 3184547 INFO (jetty-launcher-3718-thread-8) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3184547 INFO (jetty-launcher-3718-thread-8) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3184548 INFO (jetty-launcher-3718-thread-8) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-05-30T22:28:20.488Z
[junit4] 2> 3184549 INFO (jetty-launcher-3718-thread-4) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 3184552 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15c5b7961010001, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 3184556 INFO (jetty-launcher-3718-thread-6) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5e56b55b{/solr,null,AVAILABLE}
[junit4] 2> 3184556 INFO (jetty-launcher-3718-thread-6) [ ] o.e.j.s.AbstractConnector Started ServerConnector@70a2492c{HTTP/1.1,[http/1.1]}{127.0.0.1:42620}
[junit4] 2> 3184557 INFO (jetty-launcher-3718-thread-6) [ ] o.e.j.s.Server Started @3187963ms
[junit4] 2> 3184557 INFO (jetty-launcher-3718-thread-6) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42620}
[junit4] 2> 3184557 ERROR (jetty-launcher-3718-thread-6) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3184557 INFO (jetty-launcher-3718-thread-6) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 3184557 INFO (jetty-launcher-3718-thread-6) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3184557 INFO (jetty-launcher-3718-thread-6) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3184557 INFO (jetty-launcher-3718-thread-6) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-05-30T22:28:20.498Z
[junit4] 2> 3184557 INFO (jetty-launcher-3718-thread-7) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52523/solr
[junit4] 2> 3184563 INFO (jetty-launcher-3718-thread-9) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@44e0f0e3{/solr,null,AVAILABLE}
[junit4] 2> 3184565 INFO (jetty-launcher-3718-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@77abec4f{/solr,null,AVAILABLE}
[junit4] 2> 3184564 INFO (jetty-launcher-3718-thread-5) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@36974182{/solr,null,AVAILABLE}
[junit4] 2> 3184568 INFO (jetty-launcher-3718-thread-9) [ ] o.e.j.s.AbstractConnector Started ServerConnector@446ffa{HTTP/1.1,[http/1.1]}{127.0.0.1:35532}
[junit4] 2> 3184563 INFO (jetty-launcher-3718-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5e38b3ee{/solr,null,AVAILABLE}
[junit4] 2> 3184568 INFO (jetty-launcher-3718-thread-9) [ ] o.e.j.s.Server Started @3187974ms
[junit4] 2> 3184568 INFO (jetty-launcher-3718-thread-9) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35532}
[junit4] 2> 3184568 INFO (jetty-launcher-3718-thread-5) [ ] o.e.j.s.AbstractConnector Started ServerConnector@64fb14f6{HTTP/1.1,[http/1.1]}{127.0.0.1:54757}
[junit4] 2> 3184568 INFO (jetty-launcher-3718-thread-5) [ ] o.e.j.s.Server Started @3187974ms
[junit4] 2> 3184568 INFO (jetty-launcher-3718-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@cd5992d{HTTP/1.1,[http/1.1]}{127.0.0.1:33709}
[junit4] 2> 3184568 ERROR (jetty-launcher-3718-thread-9) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3184568 INFO (jetty-launcher-3718-thread-1) [ ] o.e.j.s.Server Started @3187974ms
[junit4] 2> 3184568 INFO (jetty-launcher-3718-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33709}
[junit4] 2> 3184568 INFO (jetty-launcher-3718-thread-9) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 3184568 INFO (jetty-launcher-3718-thread-9) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3184568 INFO (jetty-launcher-3718-thread-9) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3184568 ERROR (jetty-launcher-3718-thread-1) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3184563 INFO (jetty-launcher-3718-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3699e939{/solr,null,AVAILABLE}
[junit4] 2> 3184569 INFO (jetty-launcher-3718-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 3184570 INFO (jetty-launcher-3718-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3184570 INFO (jetty-launcher-3718-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3184570 INFO (jetty-launcher-3718-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-05-30T22:28:20.511Z
[junit4] 2> 3184570 INFO (jetty-launcher-3718-thread-4) [ ] o.e.j.s.AbstractConnector Started ServerConnector@402eed08{HTTP/1.1,[http/1.1]}{127.0.0.1:57617}
[junit4] 2> 3184570 INFO (jetty-launcher-3718-thread-4) [ ] o.e.j.s.Server Started @3187976ms
[junit4] 2> 3184570 INFO (jetty-launcher-3718-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=57617}
[junit4] 2> 3184570 ERROR (jetty-launcher-3718-thread-4) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3184570 INFO (jetty-launcher-3718-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 3184570 INFO (jetty-launcher-3718-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3184570 INFO (jetty-launcher-3718-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3184570 INFO (jetty-launcher-3718-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-05-30T22:28:20.511Z
[junit4] 2> 3184568 INFO (jetty-launcher-3718-thread-9) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-05-30T22:28:20.509Z
[junit4] 2> 3184568 INFO (jetty-launcher-3718-thread-5) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=54757}
[junit4] 2> 3184571 ERROR (jetty-launcher-3718-thread-5) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3184571 INFO (jetty-launcher-3718-thread-5) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 3184571 INFO (jetty-launcher-3718-thread-5) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3184571 INFO (jetty-launcher-3718-thread-5) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3184571 INFO (jetty-launcher-3718-thread-5) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-05-30T22:28:20.512Z
[junit4] 2> 3184573 INFO (jetty-launcher-3718-thread-8) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3184573 INFO (jetty-launcher-3718-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@473acb1c{/solr,null,AVAILABLE}
[junit4] 2> 3184574 INFO (jetty-launcher-3718-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@52ff47ed{HTTP/1.1,[http/1.1]}{127.0.0.1:39605}
[junit4] 2> 3184574 INFO (jetty-launcher-3718-thread-3) [ ] o.e.j.s.Server Started @3187980ms
[junit4] 2> 3184576 INFO (jetty-launcher-3718-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@335356d2{HTTP/1.1,[http/1.1]}{127.0.0.1:35746}
[junit4] 2> 3184576 INFO (jetty-launcher-3718-thread-2) [ ] o.e.j.s.Server Started @3187983ms
[junit4] 2> 3184576 INFO (jetty-launcher-3718-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35746}
[junit4] 2> 3184577 ERROR (jetty-launcher-3718-thread-2) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3184577 INFO (jetty-launcher-3718-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 3184577 INFO (jetty-launcher-3718-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3184577 INFO (jetty-launcher-3718-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3184577 INFO (jetty-launcher-3718-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-05-30T22:28:20.518Z
[junit4] 2> 3184583 INFO (jetty-launcher-3718-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=39605}
[junit4] 2> 3184584 INFO (jetty-launcher-3718-thread-6) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3184584 ERROR (jetty-launcher-3718-thread-3) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3184584 INFO (jetty-launcher-3718-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 3184584 INFO (jetty-launcher-3718-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3184584 INFO (jetty-launcher-3718-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3184584 INFO (jetty-launcher-3718-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-05-30T22:28:20.525Z
[junit4] 2> 3184590 INFO (jetty-launcher-3718-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3184590 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15c5b7961010002, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 3184595 INFO (jetty-launcher-3718-thread-10) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2d47b56a{/solr,null,AVAILABLE}
[junit4] 2> 3184596 INFO (jetty-launcher-3718-thread-10) [ ] o.e.j.s.AbstractConnector Started ServerConnector@5308fe1f{HTTP/1.1,[http/1.1]}{127.0.0.1:52623}
[junit4] 2> 3184596 INFO (jetty-launcher-3718-thread-10) [ ] o.e.j.s.Server Started @3188002ms
[junit4] 2> 3184596 INFO (jetty-launcher-3718-thread-10) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=52623}
[junit4] 2> 3184596 ERROR (jetty-launcher-3718-thread-10) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3184596 INFO (jetty-launcher-3718-thread-10) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 3184596 INFO (jetty-launcher-3718-thread-10) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3184596 INFO (jetty-launcher-3718-thread-10) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3184596 INFO (jetty-launcher-3718-thread-10) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-05-30T22:28:20.537Z
[junit4] 2> 3184598 INFO (jetty-launcher-3718-thread-9) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3184598 INFO (jetty-launcher-3718-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3184608 INFO (jetty-launcher-3718-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3184609 INFO (jetty-launcher-3718-thread-10) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3184610 INFO (jetty-launcher-3718-thread-6) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52523/solr
[junit4] 2> 3184620 INFO (jetty-launcher-3718-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3184627 INFO (jetty-launcher-3718-thread-5) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3184630 INFO (jetty-launcher-3718-thread-9) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52523/solr
[junit4] 2> 3184630 INFO (jetty-launcher-3718-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52523/solr
[junit4] 2> 3184636 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15c5b796101000d, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 3184658 INFO (jetty-launcher-3718-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52523/solr
[junit4] 2> 3184658 INFO (jetty-launcher-3718-thread-10) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52523/solr
[junit4] 2> 3184667 INFO (jetty-launcher-3718-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52523/solr
[junit4] 2> 3184668 INFO (jetty-launcher-3718-thread-8) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52523/solr
[junit4] 2> 3184669 INFO (jetty-launcher-3718-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52523/solr
[junit4] 2> 3184672 INFO (jetty-launcher-3718-thread-5) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:52523/solr
[junit4] 2> 3184680 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15c5b7961010014, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 3184685 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15c5b7961010015, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 3184711 INFO (jetty-launcher-3718-thread-6) [n:127.0.0.1:42620_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3184713 INFO (jetty-launcher-3718-thread-6) [n:127.0.0.1:42620_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42620_solr
[junit4] 2> 3184715 INFO (jetty-launcher-3718-thread-7) [n:127.0.0.1:48197_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3184716 INFO (jetty-launcher-3718-thread-6) [n:127.0.0.1:42620_solr ] o.a.s.c.Overseer Overseer (id=98053868771934225-127.0.0.1:42620_solr-n_0000000000) starting
[junit4] 2> 3184717 INFO (jetty-launcher-3718-thread-2) [n:127.0.0.1:35746_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3184725 INFO (jetty-launcher-3718-thread-7) [n:127.0.0.1:48197_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48197_solr
[junit4] 2> 3184727 INFO (jetty-launcher-3718-thread-9) [n:127.0.0.1:35532_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3184739 INFO (jetty-launcher-3718-thread-2) [n:127.0.0.1:35746_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35746_solr
[junit4] 2> 3184741 INFO (jetty-launcher-3718-thread-1) [n:127.0.0.1:33709_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3184741 INFO (jetty-launcher-3718-thread-8) [n:127.0.0.1:35599_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3184741 INFO (jetty-launcher-3718-thread-4) [n:127.0.0.1:57617_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3184745 INFO (zkCallback-3767-thread-1-processing-n:127.0.0.1:35746_solr) [n:127.0.0.1:35746_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3184745 INFO (zkCallback-3762-thread-1-processing-n:127.0.0.1:35532_solr) [n:127.0.0.1:35532_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3184745 INFO (zkCallback-3749-thread-1-processing-n:127.0.0.1:42620_solr) [n:127.0.0.1:42620_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3184745 INFO (zkCallback-3731-thread-1-processing-n:127.0.0.1:48197_solr) [n:127.0.0.1:48197_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3184745 INFO (zkCallback-3766-thread-1-processing-n:127.0.0.1:33709_solr) [n:127.0.0.1:33709_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3184746 INFO (zkCallback-3769-thread-1-processing-n:127.0.0.1:54757_solr) [n:127.0.0.1:54757_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3184746 INFO (zkCallback-3764-thread-1-processing-n:127.0.0.1:57617_solr) [n:127.0.0.1:57617_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3184747 INFO (jetty-launcher-3718-thread-5) [n:127.0.0.1:54757_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3184747 INFO (jetty-launcher-3718-thread-9) [n:127.0.0.1:35532_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35532_solr
[junit4] 2> 3184748 INFO (zkCallback-3769-thread-1-processing-n:127.0.0.1:54757_solr) [n:127.0.0.1:54757_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 3184748 INFO (zkCallback-3731-thread-1-processing-n:127.0.0.1:48197_solr) [n:127.0.0.1:48197_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 3184749 INFO (zkCallback-3764-thread-1-processing-n:127.0.0.1:57617_solr) [n:127.0.0.1:57617_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 3184750 INFO (zkCallback-3766-thread-1-processing-n:127.0.0.1:33709_solr) [n:127.0.0.1:33709_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 3184750 INFO (zkCallback-3762-thread-1-processing-n:127.0.0.1:35532_solr) [n:127.0.0.1:35532_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 3184750 INFO (zkCallback-3767-thread-1-processing-n:127.0.0.1:35746_solr) [n:127.0.0.1:35746_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 3184751 INFO (zkCallback-3731-thread-1-processing-n:127.0.0.1:48197_solr) [n:127.0.0.1:48197_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 3184751 INFO (zkCallback-3769-thread-1-processing-n:127.0.0.1:54757_solr) [n:127.0.0.1:54757_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 3184751 INFO (jetty-launcher-3718-thread-5) [n:127.0.0.1:54757_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54757_solr
[junit4] 2> 3184752 INFO (zkCallback-3764-thread-1-processing-n:127.0.0.1:57617_solr) [n:127.0.0.1:57617_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 3184752 INFO (zkCallback-3766-thread-1-processing-n:127.0.0.1:33709_solr) [n:127.0.0.1:33709_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 3184753 INFO (zkCallback-3731-thread-1-processing-n:127.0.0.1:48197_solr) [n:127.0.0.1:48197_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 3184754 INFO (jetty-launcher-3718-thread-3) [n:127.0.0.1:39605_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 3184756 INFO (jetty-launcher-3718-thread-3) [n:127.0.0.1:39605_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3184758 INFO (zkCallback-3769-thread-1-processing-n:127.0.0.1:54757_solr) [n:127.0.0.1:54757_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 3184771 INFO (zkCallback-3767-thread-1-processing-n:127.0.0.1:35746_solr) [n:127.0.0.1:35746_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (4)
[junit4] 2> 3184771 INFO (jetty-launcher-3718-thread-4) [n:127.0.0.1:57617_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57617_solr
[junit4] 2> 3184772 INFO (zkCallback-3762-thread-1-processing-n:127.0.0.1:35532_solr) [n:127.0.0.1:35532_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (4)
[junit4] 2> 3184772 INFO (jetty-launcher-3718-thread-1) [n:127.0.0.1:33709_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33709_solr
[junit4] 2> 3184773 INFO (jetty-launcher-3718-thread-3) [n:127.0.0.1:39605_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39605_solr
[junit4] 2> 3184773 INFO (zkCallback-3764-thread-1-processing-n:127.0.0.1:57617_solr) [n:127.0.0.1:57617_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 3184773 INFO (zkCallback-3766-thread-1-processing-n:127.0.0.1:33709_solr) [n:127.0.0.1:33709_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 3184774 INFO (zkCallback-3749-thread-1-processing-n:127.0.0.1:42620_solr) [n:127.0.0.1:42620_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (4)
[junit4] 2> 3184775 INFO (zkCallback-3765-thread-1-processing-n:127.0.0.1:35599_solr) [n:127.0.0.1:35599_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (4)
[junit4] 2> 3184777 INFO (jetty-launcher-3718-thread-8) [n:127.0.0.1:35599_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3184777 INFO (zkCallback-3762-thread-1-processing-n:127.0.0.1:35532_solr) [n:127.0.0.1:35532_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (6)
[junit4] 2> 3184777 INFO (zkCallback-3731-thread-1-processing-n:127.0.0.1:48197_solr) [n:127.0.0.1:48197_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (6)
[junit4] 2> 3184778 INFO (zkCallback-3766-thread-1-processing-n:127.0.0.1:33709_solr) [n:127.0.0.1:33709_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (6)
[junit4] 2> 3184778 INFO (zkCallback-3749-thread-1-processing-n:127.0.0.1:42620_solr) [n:127.0.0.1:42620_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (6)
[junit4] 2> 3184778 INFO (zkCallback-3769-thread-1-processing-n:127.0.0.1:54757_solr) [n:127.0.0.1:54757_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (7)
[junit4] 2> 3184778 INFO (zkCallback-3767-thread-1-processing-n:127.0.0.1:35746_solr) [n:127.0.0.1:35746_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (7)
[junit4] 2> 3184778 INFO (zkCallback-3765-thread-1-processing-n:127.0.0.1:35599_solr) [n:127.0.0.1:35599_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (7)
[junit4] 2> 3184778 INFO (zkCallback-3764-thread-1-processing-n:127.0.0.1:57617_solr) [n:127.0.0.1:57617_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (7)
[junit4] 2> 3184779 INFO (zkCallback-3762-thread-1-processing-n:127.0.0.1:35532_solr) [n:127.0.0.1:35532_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (7)
[junit4] 2> 3184780 INFO (jetty-launcher-3718-thread-10) [n:127.0.0.1:52623_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (7)
[junit4] 2> 3184781 INFO (jetty-launcher-3718-thread-8) [n:127.0.0.1:35599_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35599_solr
[junit4] 2> 3184782 INFO (zkCallback-3731-thread-1-processing-n:127.0.0.1:48197_solr) [n:127.0.0.1:48197_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (7)
[junit4] 2> 3184783 INFO (zkCallback-3765-thread-1-processing-n:127.0.0.1:35599_solr) [n:127.0.0.1:35599_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (7) -> (8)
[junit4] 2> 3184783 INFO (zkCallback-3762-thread-1-processing-n:127.0.0.1:35532_solr) [n:127.0.0.1:35532_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (7) -> (8)
[junit4] 2> 3184786 INFO (zkCallback-3766-thread-1-processing-n:127.0.0.1:33709_solr) [n:127.0.0.1:33709_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (7)
[junit4] 2> 3184788 INFO (zkCallback-3764-thread-1-processing-n:127.0.0.1:57617_solr) [n:127.0.0.1:57617_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (7) -> (8)
[junit4] 2> 3184788 INFO (zkCallback-3769-thread-1-processing-n:127.0.0.1:54757_solr) [n:127.0.0.1:54757_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (7) -> (8)
[junit4] 2> 3184788 INFO (zkCallback-3767-thread-1-processing-n:127.0.0.1:35746_solr) [n:127.0.0.1:35746_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (7) -> (8)
[junit4] 2> 3184788 INFO (zkCallback-3731-thread-1-processing-n:127.0.0.1:48197_solr) [n:127.0.0.1:48197_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (7) -> (8)
[junit4] 2> 3184788 INFO (zkCallback-3766-thread-1-processing-n:127.0.0.1:33709_solr) [n:127.0.0.1:33709_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (7) -> (8)
[junit4] 2> 3184804 INFO (zkCallback-3768-thread-1-processing-n:127.0.0.1:39605_solr) [n:127.0.0.1:39605_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (8)
[junit4] 2> 3184806 INFO (zkCallback-3770-thread-1-processing-n:127.0.0.1:52623_solr) [n:127.0.0.1:52623_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (7) -> (8)
[junit4] 2> 3184847 INFO (zkCallback-3749-thread-2-processing-n:127.0.0.1:42620_solr) [n:127.0.0.1:42620_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (8)
[junit4] 2> 3184848 INFO (jetty-launcher-3718-thread-10) [n:127.0.0.1:52623_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3184849 INFO (jetty-launcher-3718-thread-6) [n:127.0.0.1:42620_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42620_solr
[junit4] 2> 3184854 INFO (zkCallback-3768-thread-1-processing-n:127.0.0.1:39605_solr) [n:127.0.0.1:39605_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (8) -> (9)
[junit4] 2> 3184854 INFO (zkCallback-3765-thread-1-processing-n:127.0.0.1:35599_solr) [n:127.0.0.1:35599_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (8) -> (9)
[junit4] 2> 3184854 INFO (zkCallback-3767-thread-1-processing-n:127.0.0.1:35746_solr) [n:127.0.0.1:35746_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (8) -> (9)
[junit4] 2> 3184854 INFO (zkCallback-3749-thread-2-processing-n:127.0.0.1:42620_solr) [n:127.0.0.1:42620_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (8) -> (9)
[junit4] 2> 3184854 INFO (zkCallback-3766-thread-1-processing-n:127.0.0.1:33709_solr) [n:127.0.0.1:33709_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (8) -> (9)
[junit4] 2> 3184854 INFO (zkCallback-3770-thread-1-processing-n:127.0.0.1:52623_solr) [n:127.0.0.1:52623_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (8) -> (9)
[junit4] 2> 3184854 INFO (zkCallback-3731-thread-1-processing-n:127.0.0.1:48197_solr) [n:127.0.0.1:48197_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (8) -> (9)
[junit4] 2> 3184854 INFO (zkCallback-3764-thread-1-processing-n:127.0.0.1:57617_solr) [n:127.0.0.1:57617_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (8) -> (9)
[junit4] 2> 3184854 INFO (zkCallback-3769-thread-1-processing-n:127.0.0.1:54757_solr) [n:127.0.0.1:54757_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (8) -> (9)
[junit4] 2> 3184854 INFO (zkCallback-3762-thread-1-processing-n:127.0.0.1:35532_solr) [n:127.0.0.1:35532_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (8) -> (9)
[junit4] 2> 3184855 INFO (jetty-launcher-3718-thread-10) [n:127.0.0.1:52623_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52623_solr
[junit4] 2> 3184859 INFO (zkCallback-3770-thread-1-processing-n:127.0.0.1:52623_solr) [n:127.0.0.1:52623_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (9) -> (10)
[junit4] 2> 3184859 INFO (zkCallback-3765-thread-1-processing-n:127.0.0.1:35599_solr) [n:127.0.0.1:35599_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (9) -> (10)
[junit4] 2> 3184859 INFO (zkCallback-3769-thread-1-processing-n:127.0.0.1:54757_solr) [n:127.0.0.1:54757_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (9) -> (10)
[junit4] 2> 3184859 INFO (zkCallback-3766-thread-1-processing-n:127.0.0.1:33709_solr) [n:127.0.0.1:33709_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (9) -> (10)
[junit4] 2> 3184859 INFO (zkCallback-3764-thread-1-processing-n:127.0.0.1:57617_solr) [n:127.0.0.1:57617_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (9) -> (10)
[junit4] 2> 3184859 INFO (zkCallback-3767-thread-1-processing-n:127.0.0.1:35746_solr) [n:127.0.0.1:35746_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (9) -> (10)
[junit4] 2> 3184859 INFO (zkCallback-3731-thread-1-processing-n:127.0.0.1:48197_solr) [n:127.0.0.1:48197_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (9) -> (10)
[junit4] 2> 3184860 INFO (zkCallback-3762-thread-1-processing-n:127.0.0.1:35532_solr) [n:127.0.0.1:35532_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (9) -> (10)
[junit4] 2> 3184864 INFO (zkCallback-3768-thread-1-processing-n:127.0.0.1:39605_solr) [n:127.0.0.1:39605_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (9) -> (10)
[junit4] 2> 3184865 INFO (zkCallback-3749-thread-1-processing-n:127.0.0.1:42620_solr) [n:127.0.0.1:42620_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (9) -> (10)
[junit4] 2> 3184892 INFO (jetty-launcher-3718-thread-4) [n:127.0.0.1:57617_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_57617.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4bd1eeb4
[junit4] 2> 3184897 INFO (jetty-launcher-3718-thread-4) [n:127.0.0.1:57617_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_57617.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4bd1eeb4
[junit4] 2> 3184897 INFO (jetty-launcher-3718-thread-4) [n:127.0.0.1:57617_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_57617.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4bd1eeb4
[junit4] 2> 3184898 INFO (jetty-launcher-3718-thread-4) [n:127.0.0.1:57617_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_B086B20392B0B51A-001/tempDir-001/node7/.
[junit4] 2> 3185017 INFO (jetty-launcher-3718-thread-9) [n:127.0.0.1:35532_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35532.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4bd1eeb4
[junit4] 2> 3185022 INFO (jetty-launcher-3718-thread-9) [n:127.0.0.1:35532_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35532.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4bd1eeb4
[junit4] 2> 3185022 INFO (jetty-launcher-3718-thread-9) [n:127.0.0.1:35532_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35532.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4bd1eeb4
[junit4] 2> 3185035 INFO (jetty-launcher-3718-thread-9) [n:127.0.0.1:35532_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_B086B20392B0B51A-001/tempDir-001/node9/.
[junit4] 2> 3185077 INFO (jetty-launcher-3718-thread-1) [n:127.0.0.1:33709_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33709.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4bd1eeb4
[junit4] 2> 3185085 INFO (jetty-launcher-3718-thread-1) [n:127.0.0.1:33709_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33709.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4bd1eeb4
[junit4] 2> 3185085 INFO (jetty-launcher-3718-thread-1) [n:127.0.0.1:33709_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33709.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4bd1eeb4
[junit4] 2> 3185096 INFO (jetty-launcher-3718-thread-1) [n:127.0.0.1:33709_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_B086B20392B0B51A-001/tempDir-001/node4/.
[junit4] 2> 3185108 INFO (jetty-launcher-3718-thread-3) [n:127.0.0.1:39605_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39605.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4bd1eeb4
[junit4] 2> 3185112 INFO (jetty-launcher-3718-thread-8) [n:127.0.0.1:35599_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35599.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4bd1eeb4
[junit4] 2> 3185129 INFO (jetty-launcher-3718-thread-8) [n:127.0.0.1:35599_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35599.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4bd1eeb4
[junit4] 2> 3185154 INFO (jetty-launcher-3718-thread-8) [n:127.0.0.1:35599_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35599.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4bd1eeb4
[junit4] 2> 3185155 INFO (jetty-launcher-3718-thread-8) [n:127.0.0.1:35599_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_B086B20392B0B51A-001/tempDir-001/node10/.
[junit4] 2> 3185159 INFO (jetty-launcher-3718-thread-3) [n:127.0.0.1:39605_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39605.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4bd1eeb4
[junit4] 2> 3185159 INFO (jetty-launcher-3718-thread-3) [n:127.0.0.1:39605_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39605.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4bd1eeb4
[junit4] 2> 3185160 INFO (jetty-launcher-3718-thread-3) [n:127.0.0.1:39605_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_B086B20392B0B51A-001/tempDir-001/node6/.
[junit4] 2> 3185180 INFO (jetty-launcher-3718-thread-10) [n:127.0.0.1:52623_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_52623.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4bd1eeb4
[junit4] 2> 3185187 INFO (jetty-launcher-3718-thread-10) [n:127.0.0.1:52623_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_52623.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4bd1eeb4
[junit4] 2> 3185187 INFO (jetty-launcher-3718-thread-10) [n:127.0.0.1:52623_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_52623.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4bd1eeb4
[junit4] 2> 3185192 INFO (jetty-launcher-3718-thread-10) [n:127.0.0.1:52623_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_B086B20392B0B51A-001/tempDir-001/node5/.
[junit4] 2> 3185203 INFO (jetty-launcher-3718-thread-6) [n:127.0.0.1:42620_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42620.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4bd1eeb4
[junit4] 2> 3185218 INFO (jetty-launcher-3718-thread-6) [n:127.0.0.1:42620_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42620.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4bd1eeb4
[junit4] 2> 3185218 INFO (jetty-launcher-3718-thread-6) [n:127.0.0.1:42620_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42620.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4bd1eeb4
[junit4] 2> 3185226 INFO (jetty-launcher-3718-thread-6) [n:127.0.0.1:42620_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_B086B20392B0B51A-001/tempDir-001/node3/.
[junit4] 2> 3185248 INFO (jetty-launcher-3718-thread-2) [n:127.0.0.1:35746_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35746.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4bd1eeb4
[junit4] 2> 3185249 INFO (jetty-launcher-3718-thread-7) [n:127.0.0.1:48197_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48197.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4bd1eeb4
[junit4] 2> 3185257 INFO (jetty-launcher-3718-thread-7) [n:127.0.0.1:48197_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48197.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4bd1eeb4
[junit4] 2> 3185258 INFO (jetty-launcher-3718-thread-7) [n:127.0.0.1:48197_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48197.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4bd1eeb4
[junit4] 2> 3185259 INFO (jetty-launcher-3718-thread-7) [n:127.0.0.1:48197_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_B086B20392B0B51A-001/tempDir-001/node2/.
[junit4] 2> 3185267 INFO (jetty-launcher-3718-thread-2) [n:127.0.0.1:35746_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35746.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4bd1eeb4
[junit4] 2> 3185267 INFO (jetty-launcher-3718-thread-2) [n:127.0.0.1:35746_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35746.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4bd1eeb4
[junit4] 2> 3185270 INFO (jetty-launcher-3718-thread-2) [n:127.0.0.1:35746_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_B086B20392B0B51A-001/tempDir-001/node8/.
[junit4] 2> 3185291 INFO (jetty-launcher-3718-thread-5) [n:127.0.0.1:54757_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_54757.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4bd1eeb4
[junit4] 2> 3185295 INFO (jetty-launcher-3718-thread-5) [n:127.0.0.1:54757_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_54757.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4bd1eeb4
[junit4] 2> 3185296 INFO (jetty-launcher-3718-thread-5) [n:127.0.0.1:54757_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_54757.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4bd1eeb4
[junit4] 2> 3185297 INFO (jetty-launcher-3718-thread-5) [n:127.0.0.1:54757_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_B086B20392B0B51A-001/tempDir-001/node1/.
[junit4] 2> 3185890 INFO (SUITE-TestStressCloudBlindAtomicUpdates-seed#[B086B20392B0B51A]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (10)
[junit4] 2> 3185891 INFO (SUITE-TestStressCloudBlindAtomicUpdates-seed#[B086B20392B0B51A]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:52523/solr ready
[junit4] 2> 3186516 INFO (qtp141784959-22572) [n:127.0.0.1:48197_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=0&replicationFactor=2&property.config=solrconfig-tlog.xml&property.schema=schema-minimal-atomic-stress.xml&collection.configName=org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates_config-set&name=test_col&nrtReplicas=2&action=CREATE&numShards=5&tlogReplicas=0&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 3186524 INFO (OverseerThreadFactory-7181-thread-1-processing-n:127.0.0.1:42620_solr) [n:127.0.0.1:42620_solr ] o.a.s.c.CreateCollectionCmd Create collection test_col
[junit4] 2> 3186646 INFO (qtp1877161219-22621) [n:127.0.0.1:57617_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates_config-set&newCollection=true&collection=test_col&version=2&replicaType=NRT&property.config=solrconfig-tlog.xml&property.schema=schema-minimal-atomic-stress.xml&name=test_col_shard4_replica_n2&action=CREATE&numShards=5&shard=shard4&wt=javabin
[junit4] 2> 3186646 INFO (qtp1579811740-22583) [n:127.0.0.1:35599_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates_config-set&newCollection=true&collection=test_col&version=2&replicaType=NRT&property.config=solrconfig-tlog.xml&property.schema=schema-minimal-atomic-stress.xml&name=test_col_shard5_replica_n1&action=CREATE&numShards=5&shard=shard5&wt=javabin
[junit4] 2> 3186646 INFO (qtp1877161219-22621) [n:127.0.0.1:57617_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 3186646 INFO (qtp1579811740-22583) [n:127.0.0.1:35599_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 3186650 INFO (qtp1322965619-22596) [n:127.0.0.1:33709_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates_config-set&newCollection=true&collection=test_col&version=2&replicaType=NRT&property.config=solrconfig-tlog.xml&property.schema=schema-minimal-atomic-stress.xml&name=test_col_shard1_replica_n2&action=CREATE&numShards=5&shard=shard1&wt=javabin
[junit4] 2> 3186650 INFO (qtp1602882652-22609) [n:127.0.0.1:42620_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates_config-set&newCollection=true&collection=test_col&version=2&replicaType=NRT&property.config=solrconfig-tlog.xml&property.schema=schema-minimal-atomic-stress.xml&name=test_col_shard3_replica_n1&action=CREATE&numShards=5&shard=shard3&wt=javabin
[junit4] 2> 3186650 INFO (qtp641464675-22598) [n:127.0.0.1:54757_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates_config-set&newCollection=true&collection=test_col&version=2&replicaType=NRT&property.config=solrconfig-tlog.xml&property.schema=schema-minimal-atomic-stress.xml&name=test_col_shard4_replica_n1&action=CREATE&numShards=5&shard=shard4&wt=javabin
[junit4] 2> 3186650 INFO (qtp1322965619-22596) [n:127.0.0.1:33709_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 3186651 INFO (qtp1602882652-22609) [n:127.0.0.1:42620_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 3186651 INFO (qtp1173566026-22644) [n:127.0.0.1:35532_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates_config-set&newCollection=true&collection=test_col&version=2&replicaType=NRT&property.config=solrconfig-tlog.xml&property.schema=schema-minimal-atomic-stress.xml&name=test_col_shard2_replica_n2&action=CREATE&numShards=5&shard=shard2&wt=javabin
[junit4] 2> 3186651 INFO (qtp1367437767-22619) [n:127.0.0.1:35746_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates_config-set&newCollection=true&collection=test_col&version=2&replicaType=NRT&property.config=solrconfig-tlog.xml&property.schema=schema-minimal-atomic-stress.xml&name=test_col_shard1_replica_n1&action=CREATE&numShards=5&shard=shard1&wt=javabin
[junit4] 2> 3186650 INFO (qtp1679585538-22678) [n:127.0.0.1:52623_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates_config-set&newCollection=true&collection=test_col&version=2&replicaType=NRT&property.config=solrconfig-tlog.xml&property.schema=schema-minimal-atomic-stress.xml&name=test_col_shard2_replica_n1&action=CREATE&numShards=5&shard=shard2&wt=javabin
[junit4] 2> 3186652 INFO (qtp1544305040-22627) [n:127.0.0.1:39605_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates_config-set&newCollection=true&collection=test_col&version=2&replicaType=NRT&property.config=solrconfig-tlog.xml&property.schema=schema-minimal-atomic-stress.xml&name=test_col_shard3_replica_n2&action=CREATE&numShards=5&shard=shard3&wt=javabin
[junit4] 2> 3186651 INFO (qtp641464675-22598) [n:127.0.0.1:54757_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 3186653 INFO (qtp1544305040-22627) [n:127.0.0.1:39605_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 3186656 INFO (qtp1679585538-22678) [n:127.0.0.1:52623_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 3186657 INFO (qtp141784959-22574) [n:127.0.0.1:48197_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates_config-set&newCollection=true&collection=test_col&version=2&replicaType=NRT&property.config=solrconfig-tlog.xml&property.schema=schema-minimal-atomic-stress.xml&name=test_col_shard5_replica_n2&action=CREATE&numShards=5&shard=shard5&wt=javabin
[junit4] 2> 3186657 INFO (qtp141784959-22574) [n:127.0.0.1:48197_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 3186658 INFO (qtp1367437767-22619) [n:127.0.0.1:35746_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 3186658 INFO (qtp1173566026-22644) [n:127.0.0.1:35532_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 3186789 INFO (zkCallback-3764-thread-1-processing-n:127.0.0.1:57617_solr) [n:127.0.0.1:57617_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [10])
[junit4] 2> 3186789 INFO (zkCallback-3770-thread-1-processing-n:127.0.0.1:52623_solr) [n:127.0.0.1:52623_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [10])
[junit4] 2> 3186789 INFO (zkCallback-3767-thread-1-processing-n:127.0.0.1:35746_solr) [n:127.0.0.1:35746_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [10])
[junit4] 2> 3186789 INFO (zkCallback-3768-thread-1-processing-n:127.0.0.1:39605_solr) [n:127.0.0.1:39605_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [10])
[junit4] 2> 3186789 INFO (zkCallback-3731-thread-1-processing-n:127.0.0.1:48197_solr) [n:127.0.0.1:48197_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [10])
[junit4] 2> 3186789 INFO (zkCallback-3765-thread-1-processing-n:127.0.0.1:35599_solr) [n:127.0.0.1:35599_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [10])
[junit4] 2> 3186789 INFO (zkCallback-3769-thread-1-processing-n:127.0.0.1:54757_solr) [n:127.0.0.1:54757_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [10])
[junit4] 2> 3186789 INFO (zkCallback-3749-thread-2-processing-n:127.0.0.1:42620_solr) [n:127.0.0.1:42620_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [10])
[junit4] 2> 3186789 INFO (zkCallback-3762-thread-1-processing-n:127.0.0.1:35532_solr) [n:127.0.0.1:35532_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [10])
[junit4] 2> 3186789 INFO (zkCallback-3766-thread-1-processing-n:127.0.0.1:33709_solr) [n:127.0.0.1:33709_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [10])
[junit4] 2> 3187677 WARN (qtp1877161219-22621) [n:127.0.0.1:57617_solr c:test_col s:shard4 x:test_
[...truncated too long message...]
) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@698b9f44{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 3901897 INFO (TEST-TestDynamicLoading.testDynamicLoading-seed#[B086B20392B0B51A]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1a3731d4{/e_/u,null,UNAVAILABLE}
[junit4] 2> 3901898 ERROR (TEST-TestDynamicLoading.testDynamicLoading-seed#[B086B20392B0B51A]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 3901900 INFO (TEST-TestDynamicLoading.testDynamicLoading-seed#[B086B20392B0B51A]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:49395 49395
[junit4] 2> 3902082 INFO (Thread-91432) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:49395 49395
[junit4] 2> 3902084 WARN (Thread-91432) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 15 /solr/configs/conf1
[junit4] 2> 6 /solr/aliases.json
[junit4] 2> 5 /solr/security.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 6 /solr/clusterstate.json
[junit4] 2> 6 /solr/clusterprops.json
[junit4] 2> 4 /solr/collections/collection1/state.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 6 /solr/live_nodes
[junit4] 2> 6 /solr/collections
[junit4] 2> 4 /solr/overseer/collection-queue-work
[junit4] 2> 3 /solr/overseer/queue
[junit4] 2> 2 /solr/overseer/queue-work
[junit4] 2>
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestDynamicLoading -Dtests.method=testDynamicLoading -Dtests.seed=B086B20392B0B51A -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=no-NO -Dtests.timezone=Asia/Tashkent -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] FAILURE 25.5s J2 | TestDynamicLoading.testDynamicLoading <<<
[junit4] > Throwable #1: java.lang.AssertionError: Could not get expected value 'org.apache.solr.core.BlobStoreTestRequestHandler' for path 'overlay/requestHandler/\/test1/class' full output: {
[junit4] > "responseHeader":{
[junit4] > "status":0,
[junit4] > "QTime":0},
[junit4] > "overlay":{
[junit4] > "znodeVersion":0,
[junit4] > "runtimeLib":{"colltest":{
[junit4] > "name":"colltest",
[junit4] > "version":1}}}}, from server: null
[junit4] > at __randomizedtesting.SeedInfo.seed([B086B20392B0B51A:68CB9F54656D10BA]:0)
[junit4] > at org.apache.solr.core.TestSolrConfigHandler.testForResponseElement(TestSolrConfigHandler.java:556)
[junit4] > at org.apache.solr.core.TestDynamicLoading.testDynamicLoading(TestDynamicLoading.java:97)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.core.TestDynamicLoading_B086B20392B0B51A-001
[junit4] 2> May 30, 2017 10:40:17 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=192, maxMBSortInHeap=5.316958581053899, sim=RandomSimilarity(queryNorm=true): {}, locale=no-NO, timezone=Asia/Tashkent
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_121 (64-bit)/cpus=4,threads=1,free=34296152,total=529530880
[junit4] 2> NOTE: All tests run in this JVM: [TestClusterStateMutator, TestPullReplica, TestJsonFacets, BadIndexSchemaTest, TestSubQueryTransformer, TestLegacyField, TestSolrCoreSnapshots, HdfsSyncSliceTest, SmileWriterTest, DocValuesNotIndexedTest, LeaderElectionContextKeyTest, TestCollectionAPIs, TestCSVResponseWriter, TestTolerantSearch, TestCoreDiscovery, RecoveryAfterSoftCommitTest, TestObjectReleaseTracker, UninvertDocValuesMergePolicyTest, DateRangeFieldTest, TlogReplayBufferedWhileIndexingTest, TestReload, UnloadDistributedZkTest, TestAuthenticationFramework, SegmentsInfoRequestHandlerTest, TestLuceneMatchVersion, TestDefaultStatsCache, TestSolrDeletionPolicy1, CollectionsAPIAsyncDistributedZkTest, TermVectorComponentDistributedTest, SolrCLIZkUtilsTest, BasicDistributedZk2Test, ConfigureRecoveryStrategyTest, DistanceUnitsTest, InfixSuggestersTest, HLLUtilTest, LukeRequestHandlerTest, TestJavabinTupleStreamParser, DistributedDebugComponentTest, SolrPluginUtilsTest, TestStandardQParsers, ChaosMonkeyNothingIsSafeTest, TestReloadAndDeleteDocs, DistributedFacetPivotLargeTest, TestIBSimilarityFactory, SimpleFacetsTest, ShowFileRequestHandlerTest, TestMissingGroups, DistributedFacetPivotSmallTest, UpdateLogTest, TestNumericRangeQuery64, SimpleMLTQParserTest, CdcrReplicationDistributedZkTest, BasicAuthStandaloneTest, MergeStrategyTest, NodeMutatorTest, RecoveryZkTest, TestXmlQParser, OverseerCollectionConfigSetProcessorTest, SpatialHeatmapFacetsTest, TestLegacyTerms, HighlighterConfigTest, ExitableDirectoryReaderTest, TestReqParamsAPI, SpellCheckCollatorWithCollapseTest, TestScoreJoinQPScore, TestSearcherReuse, TestFileDictionaryLookup, TestExportWriter, SynonymTokenizerTest, TestCloudSchemaless, PreAnalyzedUpdateProcessorTest, OverseerTest, TestFilteredDocIdSet, CustomCollectionTest, MissingSegmentRecoveryTest, TestNonDefinedSimilarityFactory, SolrIndexConfigTest, MBeansHandlerTest, PrimitiveFieldTypeTest, TestFastOutputStream, HttpSolrCallGetCoreTest, SolrMetricManagerTest, AsyncCallRequestStatusResponseTest, ReplicationFactorTest, TestSortByMinMaxFunction, BinaryUpdateRequestHandlerTest, XmlUpdateRequestHandlerTest, AtomicUpdateProcessorFactoryTest, StatsComponentTest, CoreAdminHandlerTest, HdfsUnloadDistributedZkTest, TestCloudNestedDocsSort, TestExtendedDismaxParser, OutputWriterTest, LeaderElectionTest, TestBulkSchemaConcurrent, TestConfigSetProperties, ExplicitHLLTest, TestSolrQueryParser, SolrTestCaseJ4Test, HardAutoCommitTest, TestInitQParser, ManagedSchemaRoundRobinCloudTest, TestReRankQParserPlugin, FieldMutatingUpdateProcessorTest, DirectUpdateHandlerOptimizeTest, SuggesterTSTTest, TestLMDirichletSimilarityFactory, EchoParamsTest, TestSQLHandlerNonCloud, SuggesterWFSTTest, TestIndexingPerformance, TestManagedStopFilterFactory, TestSegmentSorting, TestSolrCloudWithDelegationTokens, TestPHPSerializedResponseWriter, ChaosMonkeySafeLeaderTest, LeaderElectionIntegrationTest, BasicZkTest, FullSolrCloudDistribCmdsTest, ClusterStateUpdateTest, TestRandomFaceting, ZkSolrClientTest, TestZkChroot, TestRandomDVFaceting, ShardRoutingCustomTest, TestDistributedSearch, AutoCommitTest, TestDistributedGrouping, TestFaceting, TestRecovery, TestHashPartitioner, DistributedSpellCheckComponentTest, TestStressReorder, TestJoin, TestStressVersions, DistributedTermsComponentTest, SpellCheckComponentTest, TestGroupingSearch, TestSort, TestFiltering, HighlighterTest, CurrencyFieldXmlFileTest, SolrIndexSplitterTest, SolrRequestParserTest, TestFoldingMultitermQuery, TestStressLucene, TestTrie, NoCacheHeaderTest, TestPseudoReturnFields, TestWordDelimiterFilterFactory, DefaultValueUpdateProcessorTest, TestRemoteStreaming, DocValuesMultiTest, XsltUpdateRequestHandlerTest, CacheHeaderTest, TestWriterPerf, TestQueryTypes, TermsComponentTest, TestValueSourceCache, TermVectorComponentTest, IndexSchemaRuntimeFieldTest, TestCollationField, UpdateRequestProcessorFactoryTest, PingRequestHandlerTest, SOLR749Test, UpdateParamsTest, TestSolrIndexConfig, TestQuerySenderNoQuery, ResponseLogComponentTest, TestMergePolicyConfig, TestSolrDeletionPolicy2, MultiTermTest, TestDocSet, TestBinaryField, NumericFieldsTest, MinimalSchemaTest, TestElisionMultitermQuery, TestConfig, ExternalFileFieldSortTest, TestPostingsSolrHighlighter, TestPhraseSuggestions, TestXIncludeConfig, TestSweetSpotSimilarityFactory, TestPerFieldSimilarity, TestBM25SimilarityFactory, TimeZoneUtilsTest, ResourceLoaderTest, OpenExchangeRatesOrgProviderTest, PluginInfoTest, TestFastLRUCache, DateFieldTest, DOMUtilTest, ClusterStateTest, TestSolrJ, TestUtils, TestDocumentBuilder, ZkNodePropsTest, FileUtilsTest, CircularListTest, TestRTGBase, CursorPagingTest, DistributedIntervalFacetingTest, TestCursorMarkWithoutUniqueKey, TestDistributedMissingSort, TestHighlightDedupGrouping, TestSimpleTrackingShardHandler, ChaosMonkeySafeLeaderWithPullReplicasTest, ConcurrentDeleteAndCreateCollectionTest, ForceLeaderTest, SSLMigrationTest, SaslZkACLProviderTest, SharedFSAutoReplicaFailoverTest, TestCloudDeleteByQuery, TestCloudRecovery, TestConfigSetsAPIZkFailure, TestCryptoKeys, TestDownShardTolerantSearch, TestExclusionRuleCollectionAccess, TestLeaderElectionWithEmptyReplica, TestLocalFSCloudBackupRestore, TestMiniSolrCloudCluster, TestRandomFlRTGCloud, TestStressLiveNodes, TriLevelCompositeIdRoutingTest, VMParamsZkACLAndCredentialsProvidersTest, HdfsBasicDistributedZk2Test, HdfsCollectionsAPIDistributedZkTest, HdfsWriteToMultipleCollectionsTest, StressHdfsTest, TestConfigSetImmutable, TestDynamicLoading]
[junit4] Completed [574/724 (2!)] on J2 in 25.81s, 1 test, 1 failure <<< FAILURES!
[...truncated 43824 lines...]
[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1325 - Still
unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1325/
3 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestTlogReplica
Error Message:
ObjectTracker found 5 object(s) that were not released!!! [MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, RawDirectoryWrapper, MockDirectoryWrapper] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347) at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:361) at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:721) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:948) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:855) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:979) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:914) at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:91) at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384) at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:178) at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:747) at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:728) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:509) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:374) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:318) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.eclipse.jetty.server.Server.handle(Server.java:534) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.core.SolrCore at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1033) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:855) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:979) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:914) at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:91) at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384) at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:178) at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:747) at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:728) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:509) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:374) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:318) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.eclipse.jetty.server.Server.handle(Server.java:534) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347) at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:92) at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:753) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:948) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:855) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:979) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:914) at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:91) at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384) at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:178) at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:747) at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:728) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:509) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:374) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:318) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.eclipse.jetty.server.Server.handle(Server.java:534) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.RawDirectoryWrapper at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347) at org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:490) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:942) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:855) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:979) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:914) at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:91) at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384) at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:178) at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:747) at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:728) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:509) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:374) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:318) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.eclipse.jetty.server.Server.handle(Server.java:534) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:480) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:328) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:419) at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$12(ReplicationHandler.java:1183) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
java.lang.AssertionError: ObjectTracker found 5 object(s) that were not released!!! [MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, RawDirectoryWrapper, MockDirectoryWrapper]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:361)
at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:721)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:948)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:855)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:979)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:914)
at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:91)
at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384)
at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389)
at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:178)
at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:747)
at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:728)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:509)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:374)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:318)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:534)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.core.SolrCore
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1033)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:855)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:979)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:914)
at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:91)
at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384)
at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389)
at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:178)
at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:747)
at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:728)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:509)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:374)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:318)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:534)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:92)
at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:753)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:948)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:855)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:979)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:914)
at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:91)
at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384)
at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389)
at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:178)
at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:747)
at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:728)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:509)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:374)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:318)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:534)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.RawDirectoryWrapper
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
at org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:490)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:942)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:855)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:979)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:914)
at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:91)
at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384)
at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389)
at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:178)
at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:747)
at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:728)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:509)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:374)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:318)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:534)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:480)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:328)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:419)
at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$12(ReplicationHandler.java:1183)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([3E5054D4A6DFE15A]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:295)
at sun.reflect.GeneratedMethodAccessor35.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.hdfs.HdfsUnloadDistributedZkTest.test
Error Message:
Error from server at http://127.0.0.1:44855/cssb/sf: Error CREATEing SolrCore 'test_unload_shard_and_collection_2': Unable to create core [test_unload_shard_and_collection_2] Caused by: Direct buffer memory
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:44855/cssb/sf: Error CREATEing SolrCore 'test_unload_shard_and_collection_2': Unable to create core [test_unload_shard_and_collection_2] Caused by: Direct buffer memory
at __randomizedtesting.SeedInfo.seed([3E5054D4A6DFE15A:B6046B0E08238CA2]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:592)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:230)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:219)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at org.apache.solr.cloud.UnloadDistributedZkTest.testUnloadShardAndCollection(UnloadDistributedZkTest.java:133)
at org.apache.solr.cloud.UnloadDistributedZkTest.test(UnloadDistributedZkTest.java:70)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.core.TestDynamicLoading.testDynamicLoading
Error Message:
Could not get expected value 'org.apache.solr.core.BlobStoreTestRequestHandler' for path 'overlay/requestHandler/\/test1/class' full output: { "responseHeader":{ "status":0, "QTime":0}, "overlay":{ "znodeVersion":0, "runtimeLib":{"colltest":{ "name":"colltest", "version":1}}}}, from server: null
Stack Trace:
java.lang.AssertionError: Could not get expected value 'org.apache.solr.core.BlobStoreTestRequestHandler' for path 'overlay/requestHandler/\/test1/class' full output: {
"responseHeader":{
"status":0,
"QTime":0},
"overlay":{
"znodeVersion":0,
"runtimeLib":{"colltest":{
"name":"colltest",
"version":1}}}}, from server: null
at __randomizedtesting.SeedInfo.seed([3E5054D4A6DFE15A:E61D7983510244FA]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.core.TestSolrConfigHandler.testForResponseElement(TestSolrConfigHandler.java:556)
at org.apache.solr.core.TestDynamicLoading.testDynamicLoading(TestDynamicLoading.java:97)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 11567 lines...]
[junit4] Suite: org.apache.solr.cloud.hdfs.HdfsUnloadDistributedZkTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_3E5054D4A6DFE15A-001/init-core-data-001
[junit4] 2> 303423 WARN (SUITE-HdfsUnloadDistributedZkTest-seed#[3E5054D4A6DFE15A]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=5 numCloses=5
[junit4] 2> 303424 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[3E5054D4A6DFE15A]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields
[junit4] 2> 303426 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[3E5054D4A6DFE15A]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
[junit4] 2> 303426 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[3E5054D4A6DFE15A]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /cssb/sf
[junit4] 2> 304221 WARN (SUITE-HdfsUnloadDistributedZkTest-seed#[3E5054D4A6DFE15A]-worker) [ ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 305628 WARN (SUITE-HdfsUnloadDistributedZkTest-seed#[3E5054D4A6DFE15A]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 305851 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[3E5054D4A6DFE15A]-worker) [ ] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
[junit4] 2> 305880 WARN (SUITE-HdfsUnloadDistributedZkTest-seed#[3E5054D4A6DFE15A]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 305952 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[3E5054D4A6DFE15A]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 306002 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[3E5054D4A6DFE15A]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_44836_hdfs____.ills8r/webapp
[junit4] 2> 307055 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[3E5054D4A6DFE15A]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44836
[junit4] 2> 308534 WARN (SUITE-HdfsUnloadDistributedZkTest-seed#[3E5054D4A6DFE15A]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 308540 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[3E5054D4A6DFE15A]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 308553 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[3E5054D4A6DFE15A]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_58394_datanode____j55pwy/webapp
[junit4] 2> 309124 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[3E5054D4A6DFE15A]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:58394
[junit4] 2> 309624 WARN (SUITE-HdfsUnloadDistributedZkTest-seed#[3E5054D4A6DFE15A]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 309626 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[3E5054D4A6DFE15A]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 309641 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[3E5054D4A6DFE15A]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_45877_datanode____6ut74u/webapp
[junit4] 2> 310055 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[3E5054D4A6DFE15A]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45877
[junit4] 2> 311091 INFO (IPC Server handler 7 on 49148) [ ] BlockStateChange BLOCK* processReport: from storage DS-ec379af6-86c4-4d64-b826-aa6396dc3fa2 node DatanodeRegistration(127.0.0.1:57882, datanodeUuid=23ce35cf-bac7-45ee-8a52-15768f3c24b2, infoPort=41899, infoSecurePort=0, ipcPort=39654, storageInfo=lv=-56;cid=testClusterID;nsid=1356251657;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 311091 INFO (IPC Server handler 4 on 49148) [ ] BlockStateChange BLOCK* processReport: from storage DS-04583019-d2c5-4130-b2de-bf7cab0d053d node DatanodeRegistration(127.0.0.1:59614, datanodeUuid=e8ab60e0-8bdd-49d1-bfa7-5b57411297a1, infoPort=38465, infoSecurePort=0, ipcPort=36938, storageInfo=lv=-56;cid=testClusterID;nsid=1356251657;c=0), blocks: 0, hasStaleStorage: true, processing time: 3 msecs
[junit4] 2> 311092 INFO (IPC Server handler 7 on 49148) [ ] BlockStateChange BLOCK* processReport: from storage DS-91885a43-4bfb-448b-88b5-1290fdced416 node DatanodeRegistration(127.0.0.1:57882, datanodeUuid=23ce35cf-bac7-45ee-8a52-15768f3c24b2, infoPort=41899, infoSecurePort=0, ipcPort=39654, storageInfo=lv=-56;cid=testClusterID;nsid=1356251657;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 311092 INFO (IPC Server handler 4 on 49148) [ ] BlockStateChange BLOCK* processReport: from storage DS-a95327fe-089d-49bb-9ce9-d8a4f9ef8884 node DatanodeRegistration(127.0.0.1:59614, datanodeUuid=e8ab60e0-8bdd-49d1-bfa7-5b57411297a1, infoPort=38465, infoSecurePort=0, ipcPort=36938, storageInfo=lv=-56;cid=testClusterID;nsid=1356251657;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 311165 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 311165 INFO (Thread-729) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 311166 INFO (Thread-729) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 311169 ERROR (Thread-729) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 311266 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.a.s.c.ZkTestServer start zk server on port:37684
[junit4] 2> 311296 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 311298 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 311300 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 311302 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 311303 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 311307 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 311309 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 311310 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 311315 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 311316 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 311317 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 311319 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
[junit4] 2> 311433 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_3E5054D4A6DFE15A-001/control-001/cores/collection1
[junit4] 2> 311435 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 311454 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@702447ca{/cssb/sf,null,AVAILABLE}
[junit4] 2> 311456 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@13309b7d{HTTP/1.1,[http/1.1]}{127.0.0.1:51276}
[junit4] 2> 311456 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.e.j.s.Server Started @314421ms
[junit4] 2> 311456 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:49148/hdfs__localhost_49148__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-master_checkout_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.HdfsUnloadDistributedZkTest_3E5054D4A6DFE15A-001_tempDir-002_control_data, replicaType=NRT, hostContext=/cssb/sf, hostPort=51276, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_3E5054D4A6DFE15A-001/control-001/cores}
[junit4] 2> 311457 ERROR (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 311457 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 311457 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 311457 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 311457 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-06-14T23:12:48.620Z
[junit4] 2> 311461 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 311461 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_3E5054D4A6DFE15A-001/control-001/solr.xml
[junit4] 2> 311469 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@52d4fb33, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 311472 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37684/solr
[junit4] 2> 311494 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [n:127.0.0.1:51276_cssb%2Fsf ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 311495 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [n:127.0.0.1:51276_cssb%2Fsf ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:51276_cssb%2Fsf
[junit4] 2> 311496 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [n:127.0.0.1:51276_cssb%2Fsf ] o.a.s.c.Overseer Overseer (id=98138978276540420-127.0.0.1:51276_cssb%2Fsf-n_0000000000) starting
[junit4] 2> 311504 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [n:127.0.0.1:51276_cssb%2Fsf ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51276_cssb%2Fsf
[junit4] 2> 311505 INFO (zkCallback-336-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf) [n:127.0.0.1:51276_cssb%2Fsf ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 311660 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [n:127.0.0.1:51276_cssb%2Fsf ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@52d4fb33
[junit4] 2> 311667 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [n:127.0.0.1:51276_cssb%2Fsf ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@52d4fb33
[junit4] 2> 311667 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [n:127.0.0.1:51276_cssb%2Fsf ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@52d4fb33
[junit4] 2> 311669 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [n:127.0.0.1:51276_cssb%2Fsf ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_3E5054D4A6DFE15A-001/control-001/cores
[junit4] 2> 311669 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [n:127.0.0.1:51276_cssb%2Fsf ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 311672 INFO (coreLoadExecutor-1327-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 311677 INFO (OverseerStateUpdate-98138978276540420-127.0.0.1:51276_cssb%2Fsf-n_0000000000) [n:127.0.0.1:51276_cssb%2Fsf ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 312718 INFO (coreLoadExecutor-1327-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 312741 INFO (coreLoadExecutor-1327-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 312874 INFO (coreLoadExecutor-1327-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 312892 INFO (coreLoadExecutor-1327-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection, trusted=true
[junit4] 2> 312893 INFO (coreLoadExecutor-1327-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.core_node1' (registry 'solr.core.control_collection.shard1.core_node1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@52d4fb33
[junit4] 2> 312909 INFO (coreLoadExecutor-1327-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:49148/solr_hdfs_home
[junit4] 2> 312909 INFO (coreLoadExecutor-1327-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 312909 INFO (coreLoadExecutor-1327-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 312909 INFO (coreLoadExecutor-1327-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_3E5054D4A6DFE15A-001/control-001/cores/collection1], dataDir=[hdfs://localhost:49148/solr_hdfs_home/control_collection/core_node1/data/]
[junit4] 2> 312911 INFO (coreLoadExecutor-1327-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:49148/solr_hdfs_home/control_collection/core_node1/data/snapshot_metadata
[junit4] 2> 312929 INFO (coreLoadExecutor-1327-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 312930 INFO (coreLoadExecutor-1327-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 312930 INFO (coreLoadExecutor-1327-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 313584 WARN (coreLoadExecutor-1327-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection x:collection1] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
[junit4] 2> 319019 INFO (coreLoadExecutor-1327-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 319027 INFO (coreLoadExecutor-1327-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:49148/solr_hdfs_home/control_collection/core_node1/data
[junit4] 2> 319069 INFO (coreLoadExecutor-1327-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:49148/solr_hdfs_home/control_collection/core_node1/data/index
[junit4] 2> 319078 INFO (coreLoadExecutor-1327-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 319078 INFO (coreLoadExecutor-1327-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 319078 INFO (coreLoadExecutor-1327-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 319088 INFO (coreLoadExecutor-1327-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 319088 INFO (coreLoadExecutor-1327-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=13670897, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 319422 INFO (IPC Server handler 0 on 49148) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:59614 is added to blk_1073741825_1001{UCState=COMMITTED, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-04583019-d2c5-4130-b2de-bf7cab0d053d:NORMAL:127.0.0.1:59614|RBW], ReplicaUC[[DISK]DS-ec379af6-86c4-4d64-b826-aa6396dc3fa2:NORMAL:127.0.0.1:57882|RBW]]} size 72
[junit4] 2> 319422 INFO (IPC Server handler 3 on 49148) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:57882 is added to blk_1073741825_1001 size 72
[junit4] 2> 319846 WARN (coreLoadExecutor-1327-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 319908 INFO (coreLoadExecutor-1327-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 319908 INFO (coreLoadExecutor-1327-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 319908 INFO (coreLoadExecutor-1327-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 319925 INFO (coreLoadExecutor-1327-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 319925 INFO (coreLoadExecutor-1327-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 319928 INFO (coreLoadExecutor-1327-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@52999561
[junit4] 2> 320050 INFO (coreLoadExecutor-1327-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2a490451[collection1] main]
[junit4] 2> 320052 INFO (coreLoadExecutor-1327-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 320052 INFO (coreLoadExecutor-1327-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 320055 INFO (coreLoadExecutor-1327-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 320057 INFO (searcherExecutor-1328-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf x:collection1 c:control_collection) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2a490451[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 320059 INFO (coreLoadExecutor-1327-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1570223661747535872
[junit4] 2> 320069 INFO (coreZkRegister-1320-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf x:collection1 c:control_collection) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 320069 INFO (coreZkRegister-1320-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf x:collection1 c:control_collection) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 320069 INFO (coreZkRegister-1320-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf x:collection1 c:control_collection) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:51276/cssb/sf/collection1/
[junit4] 2> 320069 INFO (coreZkRegister-1320-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf x:collection1 c:control_collection) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 320070 INFO (coreZkRegister-1320-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf x:collection1 c:control_collection) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:51276/cssb/sf/collection1/ has no replicas
[junit4] 2> 320070 INFO (coreZkRegister-1320-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf x:collection1 c:control_collection) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 320075 INFO (coreZkRegister-1320-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf x:collection1 c:control_collection) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:51276/cssb/sf/collection1/ shard1
[junit4] 2> 320209 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 320210 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37684/solr ready
[junit4] 2> 320210 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 320227 INFO (coreZkRegister-1320-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf x:collection1 c:control_collection) [n:127.0.0.1:51276_cssb%2Fsf c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 320324 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_3E5054D4A6DFE15A-001/shard-1-001/cores/collection1
[junit4] 2> 320325 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_3E5054D4A6DFE15A-001/shard-1-001 of type NRT
[junit4] 2> 320326 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 320327 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@748b9c4a{/cssb/sf,null,AVAILABLE}
[junit4] 2> 320328 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3d1b8e1a{HTTP/1.1,[http/1.1]}{127.0.0.1:44855}
[junit4] 2> 320328 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.e.j.s.Server Started @323292ms
[junit4] 2> 320328 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:49148/hdfs__localhost_49148__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-master_checkout_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.HdfsUnloadDistributedZkTest_3E5054D4A6DFE15A-001_tempDir-002_jetty1, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/cssb/sf, hostPort=44855, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_3E5054D4A6DFE15A-001/shard-1-001/cores}
[junit4] 2> 320328 ERROR (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 320331 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 320331 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 320332 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 320332 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-06-14T23:12:57.495Z
[junit4] 2> 320346 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 320346 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_3E5054D4A6DFE15A-001/shard-1-001/solr.xml
[junit4] 2> 320353 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@52d4fb33, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 320356 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37684/solr
[junit4] 2> 320374 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [n:127.0.0.1:44855_cssb%2Fsf ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 320377 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [n:127.0.0.1:44855_cssb%2Fsf ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 320379 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [n:127.0.0.1:44855_cssb%2Fsf ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44855_cssb%2Fsf
[junit4] 2> 320381 INFO (zkCallback-336-thread-1-processing-n:127.0.0.1:51276_cssb%2Fsf) [n:127.0.0.1:51276_cssb%2Fsf ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 320382 INFO (zkCallback-340-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 320383 INFO (zkCallback-345-thread-1-processing-n:127.0.0.1:44855_cssb%2Fsf) [n:127.0.0.1:44855_cssb%2Fsf ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 320485 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [n:127.0.0.1:44855_cssb%2Fsf ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@52d4fb33
[junit4] 2> 320492 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [n:127.0.0.1:44855_cssb%2Fsf ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@52d4fb33
[junit4] 2> 320492 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [n:127.0.0.1:44855_cssb%2Fsf ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@52d4fb33
[junit4] 2> 320494 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [n:127.0.0.1:44855_cssb%2Fsf ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_3E5054D4A6DFE15A-001/shard-1-001/cores
[junit4] 2> 320494 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[3E5054D4A6DFE15A]) [n:127.0.0.1:44855_cssb%2Fsf ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 320495 INFO (coreLoadExecutor-1338-thread-1-processing-n:127.0.0.1:44855_cssb%2Fsf) [n:127.0.0.1:44855_cssb%2Fsf c:collection1 x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 320497 INFO (OverseerStateUpdate-98138978276540420-127.0.0.1:51276_cssb%2Fsf-n_0000000000) [n:127.0.0.1:51276_cssb%2Fsf ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 321526 INFO (coreLoadExecutor-1338-thread-1-processing-n:127.0.0.1:44855_cssb%2Fsf) [n:127.0.0.1:44855_cssb%2Fsf c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 321541 INFO (coreLoadExecutor-1338-thread-1-processing-n:127.0.0.1:44855_cssb%2Fsf) [n:127.0.0.1:44855_cssb%2Fsf c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 321655 INFO (coreLoadExecutor-1338-thread-1-processing-n:127.0.0.1:44855_cssb%2Fsf) [n:127.0.0.1:44855_cssb%2Fsf c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 321669 INFO (coreLoadExecutor-1338-thread-1-processing-n:127.0.0.1:44855_cssb%2Fsf) [n:127.0.0.1:44855_cssb%2Fsf c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1, trusted=true
[junit4] 2> 321669 INFO (coreLoadExecutor-1338-thread-1-processing-n:127.0.0.1:44855_cssb%2Fsf) [n:127.0.0.1:44855_cssb%2Fsf c:collection1 x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.core_node1' (registry 'solr.core.collection1.shard2.core_node1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@52d4fb33
[junit4] 2> 321670 INFO (coreLoadExecutor-1338-thread-1-processing-n:127.0.0.1:44855_cssb%2Fsf) [n:127.0.0.1:44855_cssb%2Fsf c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:49148/solr_hdfs_home
[junit4] 2> 321670 INFO (coreLoadExecutor-1338-thread-1-processing-n:127.0.0.1:44855_cssb%2Fsf) [n:127.0.0.1:44855_cssb%2Fsf c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 321670 INFO (coreLoadExecutor-1338-thread-1-processing-n:127.0.0.1:44855_cssb%2Fsf) [n:127.0.0.1:44855_cssb%2Fsf c:collection1 x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 321670 INFO (coreLoadExecutor-1338-thread-1-processing-n:127.0.0.1:44855_cssb%2Fsf) [n:127.0.0.1:44855_cssb%2Fsf c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_3E5054D4A6DFE15A-001/shard-1-001/cores/collection1], dataDir=[hdfs://localhost:49148/solr_hdfs_home/collection1/core_node1/data/]
[junit4] 2> 321671 INFO (coreLoadExecutor-1338-thread-1-processing-n:127.0.0.1:44855_cssb%2Fsf) [n:127.0.0.1:44855_cssb%2Fsf c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:49148/solr_hdfs_home/collection1/core_node1/data/snapshot_metadata
[junit4] 2> 321682 INFO (coreLoadExecutor-1338-thread-1-processing-n:127.0.0.1:44855_cssb%2Fsf) [n:127.0.0.1:44855_cssb%2Fsf c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 321682 INFO (coreLoadExecutor-1338-thread-1-processing-n:127.0.0.1:44855_cssb%2Fsf) [n:127.0.0.1:44855_cssb%2Fsf c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 321682 INFO (coreLoadExecutor-1338-thread-1-processing-n:127.0.0.1:44855_cssb%2Fsf) [n:127.0.0.1:44855_cssb%2Fsf c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 321694 INFO (coreLoadExecutor-1338-thread-1-processing-n:127.0.0.1:44855_cssb%2Fsf) [n:127.0.0.1:44855_cssb%2Fsf c:collection1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 321696 INFO (coreLoadExecutor-1338-thread-1-processing-n:127.0.0.1:44855_cssb%2Fsf) [n:127.0.0.1:44855_cssb%2Fsf c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:49148/solr_hdfs_home/collection1/core_node1/data
[junit4] 2> 321733 INFO (coreLoadExecutor-1338-thread-1-processing-n:127.0.0.1:44855_cssb%2Fsf) [n:127.0.0.1:44855_cssb%2Fsf c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:49148/solr_hdfs_home/collection1/core_node1/data/index
[junit4] 2> 321748 INFO (coreLoadExecutor-1338-thread-1-processing-n:127.0.0.1:44855_cssb%2Fsf) [n:127.0.0.1:44855_cssb%2Fsf c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 321749 INFO (coreLoadExecutor-1338-thread-1-processing-n:127.0.0.1:44855_cssb%2Fsf) [n:127.0.0.1:44855_cssb%2Fsf c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 321749 INFO (coreLoadExecutor-1338-thread-1-processing-n:127.0.0.1:44855_cssb%2Fsf) [n:127.0.0.1:44855_cssb%2Fsf c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 321768 INFO (coreLoadExecutor-1338-thread-1-processing-n:127.0.0.1:44855_cssb%2Fsf) [n:127.0.0.1:44855_cssb%2Fsf c:collection1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 321770 INFO (coreLoadExecutor-1338-thread-1-processing-n:127.0.0.1:44855_cssb%2Fsf) [n:127.0.0.1:44855_cssb%2Fsf c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=13670897, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 321816 INFO (IPC Server handler 5 on 49148) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:59614 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-ec379af6-86c4-4d64-b826-aa6396dc3fa2:NORMAL:127.0.0.1:57882|RBW], ReplicaUC[[DISK]DS-a95327fe-089d-49bb-9ce9-d8a4f9ef8884:NORMAL:127.0.0.1:59614|FINALIZED]]} size 0
[junit4] 2> 321817 INFO (IPC Server handler 5 on 49148) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:57882 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-a95327fe-089d-49bb-9ce9-d8a4f9ef8884:NORMAL:127.0.0.1:59614|FINALIZED], ReplicaUC[[DISK]DS-91885a43-4bfb-448b-88b5-1290fdced416:NORMAL:127.0.0.1:57882|FINALIZED]]} size 0
[junit4] 2> 321826 WARN (coreLoadExecutor-1338-thread-1-processing-n:127.0.0.1:44855_cssb%2Fsf) [n:127.0.0.1:44855_cssb%2Fsf c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 321891 INFO (coreLoadExecutor-1338-thread-1-processing-n:127.0.0.1:44855_cssb%2Fsf) [n:127.0.0.1:44855_cssb%2Fsf c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 321891 INFO (coreLoadExecutor-1338-thread-1-processing-n:127.0.0.1:44855_cssb%2Fsf) [n:127.0.0.1:44855_cssb%2Fsf c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 321891 INFO (coreLoadExecutor-1338-thread-1-processing-n:127.0.0.1:44855_cssb%2Fsf) [n:127.0.0.1:44855_cssb%2Fsf c:collection1 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 321909 INFO (coreLoadExecutor-1338-thread-1-processing-n:127.0.0.1:44855_cssb%2Fsf) [n:127.0.0.1:44855_cssb%2Fsf c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 321909 INFO (coreLoadExecutor-1338-thread-1-processing-n:127.0.0.1:44855_cssb%2Fsf) [n:127.0.0.1:44855_cssb%2Fsf c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 321912 INFO (coreLoadExecutor-1338-thread-1-processing-n:127.0.0.1:44855_cssb%2Fsf) [n:127.0.0.1:44855_cssb%2Fsf c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@4862ea40
[junit4] 2> 321932 INFO (coreLoadExecutor-1338-thread-1-processing-n:127.0.0.1:44855_cssb%2Fsf) [n:127.0.0.1:44855
[...truncated too long message...]
riter.create(SolrIndexWriter.java:92)
[junit4] > at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:753)
[junit4] > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:948)
[junit4] > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:855)
[junit4] > at org.apache.solr.core.CoreContainer.create(CoreContainer.java:979)
[junit4] > at org.apache.solr.core.CoreContainer.create(CoreContainer.java:914)
[junit4] > at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:91)
[junit4] > at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384)
[junit4] > at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389)
[junit4] > at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174)
[junit4] > at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:178)
[junit4] > at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:747)
[junit4] > at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:728)
[junit4] > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:509)
[junit4] > at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:374)
[junit4] > at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:318)
[junit4] > at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
[junit4] > at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] > at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
[junit4] > at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] > at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] > at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] > at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] > at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] > at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] > at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] > at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395)
[junit4] > at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] > at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] > at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] > at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] > at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
[junit4] > at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
[junit4] > at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.RawDirectoryWrapper
[junit4] > at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
[junit4] > at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
[junit4] > at org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:490)
[junit4] > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:942)
[junit4] > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:855)
[junit4] > at org.apache.solr.core.CoreContainer.create(CoreContainer.java:979)
[junit4] > at org.apache.solr.core.CoreContainer.create(CoreContainer.java:914)
[junit4] > at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:91)
[junit4] > at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384)
[junit4] > at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389)
[junit4] > at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174)
[junit4] > at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:178)
[junit4] > at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:747)
[junit4] > at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:728)
[junit4] > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:509)
[junit4] > at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:374)
[junit4] > at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:318)
[junit4] > at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
[junit4] > at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] > at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
[junit4] > at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] > at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] > at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] > at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] > at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] > at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] > at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] > at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395)
[junit4] > at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] > at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] > at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] > at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] > at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
[junit4] > at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
[junit4] > at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper
[junit4] > at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
[junit4] > at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
[junit4] > at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:480)
[junit4] > at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:328)
[junit4] > at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:419)
[junit4] > at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$12(ReplicationHandler.java:1183)
[junit4] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
[junit4] > at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
[junit4] > at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([3E5054D4A6DFE15A]:0)
[junit4] > at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:295)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] Completed [542/725 (3!)] on J2 in 134.95s, 13 tests, 1 failure <<< FAILURES!
[...truncated 43935 lines...]
[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1324 - Still
Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1324/
9 tests failed.
FAILED: org.apache.solr.core.TestSolrConfigHandler.testReqParams
Error Message:
Could not get expected value 'CY val' for path 'params/c' full output: {"error":{ "msg":"[Ljava.lang.String; cannot be cast to java.lang.String", "trace":"java.lang.ClassCastException: [Ljava.lang.String; cannot be cast to java.lang.String\n\tat org.apache.solr.common.params.MapSolrParams.getParams(MapSolrParams.java:39)\n\tat org.apache.solr.common.params.DefaultSolrParams.getParams(DefaultSolrParams.java:44)\n\tat org.apache.solr.common.params.MultiMapSolrParams.asMultiMap(MultiMapSolrParams.java:103)\n\tat org.apache.solr.common.params.MultiMapSolrParams.asMultiMap(MultiMapSolrParams.java:81)\n\tat org.apache.solr.request.json.RequestUtil.processParams(RequestUtil.java:119)\n\tat org.apache.solr.util.SolrPluginUtils.setDefaults(SolrPluginUtils.java:177)\n\tat org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:175)\n\tat org.apache.solr.api.ApiBag$ReqHandlerToApi.call(ApiBag.java:241)\n\tat org.apache.solr.api.V2HttpCall.execute(V2HttpCall.java:318)\n\tat org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:528)\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:374)\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:318)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)\n\tat org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)\n\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)\n\tat org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)\n\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)\n\tat org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\n\tat org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:462)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)\n\tat org.eclipse.jetty.server.Server.handle(Server.java:534)\n\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)\n\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)\n\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)\n\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)\n\tat org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)\n\tat org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)\n\tat org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)\n\tat org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)\n\tat java.lang.Thread.run(Thread.java:748)\n", "code":500}}, from server: null
Stack Trace:
java.lang.AssertionError: Could not get expected value 'CY val' for path 'params/c' full output: {"error":{
"msg":"[Ljava.lang.String; cannot be cast to java.lang.String",
"trace":"java.lang.ClassCastException: [Ljava.lang.String; cannot be cast to java.lang.String\n\tat org.apache.solr.common.params.MapSolrParams.getParams(MapSolrParams.java:39)\n\tat org.apache.solr.common.params.DefaultSolrParams.getParams(DefaultSolrParams.java:44)\n\tat org.apache.solr.common.params.MultiMapSolrParams.asMultiMap(MultiMapSolrParams.java:103)\n\tat org.apache.solr.common.params.MultiMapSolrParams.asMultiMap(MultiMapSolrParams.java:81)\n\tat org.apache.solr.request.json.RequestUtil.processParams(RequestUtil.java:119)\n\tat org.apache.solr.util.SolrPluginUtils.setDefaults(SolrPluginUtils.java:177)\n\tat org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:175)\n\tat org.apache.solr.api.ApiBag$ReqHandlerToApi.call(ApiBag.java:241)\n\tat org.apache.solr.api.V2HttpCall.execute(V2HttpCall.java:318)\n\tat org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:528)\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:374)\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:318)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)\n\tat org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)\n\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)\n\tat org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)\n\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)\n\tat org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\n\tat org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:462)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)\n\tat org.eclipse.jetty.server.Server.handle(Server.java:534)\n\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)\n\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)\n\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)\n\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)\n\tat org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)\n\tat org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)\n\tat org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)\n\tat org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)\n\tat java.lang.Thread.run(Thread.java:748)\n",
"code":500}}, from server: null
at __randomizedtesting.SeedInfo.seed([524737AC3F023EA7:256B0FA56891BA4A]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.core.TestSolrConfigHandler.testForResponseElement(TestSolrConfigHandler.java:556)
at org.apache.solr.core.TestSolrConfigHandler.testReqParams(TestSolrConfigHandler.java:665)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.handler.TestSolrConfigHandlerCloud.test
Error Message:
Could not get expected value 'CY val' for path 'params/c' full output: { "responseHeader":{ "status":500, "QTime":0}, "error":{ "msg":"[Ljava.lang.String; cannot be cast to java.lang.String", "trace":"java.lang.ClassCastException: [Ljava.lang.String; cannot be cast to java.lang.String\n\tat org.apache.solr.common.params.MapSolrParams.getParams(MapSolrParams.java:39)\n\tat org.apache.solr.common.params.MultiMapSolrParams.asMultiMap(MultiMapSolrParams.java:103)\n\tat org.apache.solr.common.params.MultiMapSolrParams.asMultiMap(MultiMapSolrParams.java:81)\n\tat org.apache.solr.request.json.RequestUtil.processParams(RequestUtil.java:119)\n\tat org.apache.solr.util.SolrPluginUtils.setDefaults(SolrPluginUtils.java:177)\n\tat org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:175)\n\tat org.apache.solr.core.SolrCore.execute(SolrCore.java:2487)\n\tat org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:722)\n\tat org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:528)\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:374)\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:318)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)\n\tat org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)\n\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)\n\tat org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)\n\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)\n\tat org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\n\tat org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:462)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)\n\tat org.eclipse.jetty.server.Server.handle(Server.java:534)\n\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)\n\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)\n\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)\n\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)\n\tat org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:202)\n\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)\n\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)\n\tat org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)\n\tat org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)\n\tat org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)\n\tat org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)\n\tat java.lang.Thread.run(Thread.java:748)\n", "code":500}}, from server: https://127.0.0.1:49280/collection1
Stack Trace:
java.lang.AssertionError: Could not get expected value 'CY val' for path 'params/c' full output: {
"responseHeader":{
"status":500,
"QTime":0},
"error":{
"msg":"[Ljava.lang.String; cannot be cast to java.lang.String",
"trace":"java.lang.ClassCastException: [Ljava.lang.String; cannot be cast to java.lang.String\n\tat org.apache.solr.common.params.MapSolrParams.getParams(MapSolrParams.java:39)\n\tat org.apache.solr.common.params.MultiMapSolrParams.asMultiMap(MultiMapSolrParams.java:103)\n\tat org.apache.solr.common.params.MultiMapSolrParams.asMultiMap(MultiMapSolrParams.java:81)\n\tat org.apache.solr.request.json.RequestUtil.processParams(RequestUtil.java:119)\n\tat org.apache.solr.util.SolrPluginUtils.setDefaults(SolrPluginUtils.java:177)\n\tat org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:175)\n\tat org.apache.solr.core.SolrCore.execute(SolrCore.java:2487)\n\tat org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:722)\n\tat org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:528)\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:374)\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:318)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)\n\tat org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)\n\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)\n\tat org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)\n\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)\n\tat org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\n\tat org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:462)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)\n\tat org.eclipse.jetty.server.Server.handle(Server.java:534)\n\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)\n\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)\n\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)\n\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)\n\tat org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:202)\n\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)\n\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)\n\tat org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)\n\tat org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)\n\tat org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)\n\tat org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)\n\tat java.lang.Thread.run(Thread.java:748)\n",
"code":500}}, from server: https://127.0.0.1:49280/collection1
at __randomizedtesting.SeedInfo.seed([524737AC3F023EA7:DA13087691FE535F]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.core.TestSolrConfigHandler.testForResponseElement(TestSolrConfigHandler.java:556)
at org.apache.solr.handler.TestSolrConfigHandlerCloud.testReqParams(TestSolrConfigHandlerCloud.java:217)
at org.apache.solr.handler.TestSolrConfigHandlerCloud.test(TestSolrConfigHandlerCloud.java:68)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.index.hdfs.CheckHdfsIndexTest
Error Message:
ObjectTracker found 1 object(s) that were not released!!! [NRTCachingDirectory] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347) at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:361) at org.apache.solr.core.SolrCore.cleanupOldIndexDirectories(SolrCore.java:3015) at org.apache.solr.core.SolrCore.close(SolrCore.java:1579) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:994) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:855) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:979) at org.apache.solr.core.CoreContainer.lambda$load$7(CoreContainer.java:611) at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
java.lang.AssertionError: ObjectTracker found 1 object(s) that were not released!!! [NRTCachingDirectory]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:361)
at org.apache.solr.core.SolrCore.cleanupOldIndexDirectories(SolrCore.java:3015)
at org.apache.solr.core.SolrCore.close(SolrCore.java:1579)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:994)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:855)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:979)
at org.apache.solr.core.CoreContainer.lambda$load$7(CoreContainer.java:611)
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([524737AC3F023EA7]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:295)
at sun.reflect.GeneratedMethodAccessor37.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.search.TestRecoveryHdfs.testLogReplay
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([524737AC3F023EA7:CC261D8EA2A9202B]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.search.TestRecoveryHdfs.testLogReplay(TestRecoveryHdfs.java:208)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.search.TestRecoveryHdfs.testRecoveryMultipleLogs
Error Message:
SolrCore 'collection1' is not available due to init failure: The max direct memory is likely too low. Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~8.388608 MB.
Stack Trace:
org.apache.solr.core.SolrCoreInitializationException: SolrCore 'collection1' is not available due to init failure: The max direct memory is likely too low. Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~8.388608 MB.
at __randomizedtesting.SeedInfo.seed([524737AC3F023EA7:EBD2AF9DB879794E]:0)
at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1388)
at org.apache.solr.util.TestHarness.getCore(TestHarness.java:250)
at org.apache.solr.search.TestRecoveryHdfs.testRecoveryMultipleLogs(TestRecoveryHdfs.java:954)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.common.SolrException: The max direct memory is likely too low. Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~8.388608 MB.
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1002)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:855)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:979)
at org.apache.solr.core.CoreContainer.lambda$load$7(CoreContainer.java:611)
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
... 1 more
Caused by: java.lang.RuntimeException: The max direct memory is likely too low. Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~8.388608 MB.
at org.apache.solr.core.HdfsDirectoryFactory.createBlockCache(HdfsDirectoryFactory.java:311)
at org.apache.solr.core.HdfsDirectoryFactory.getBlockDirectoryCache(HdfsDirectoryFactory.java:287)
at org.apache.solr.core.HdfsDirectoryFactory.create(HdfsDirectoryFactory.java:227)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:346)
at org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:490)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:942)
... 9 more
Caused by: java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Bits.java:693)
at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
at org.apache.solr.store.blockcache.BlockCache.<init>(BlockCache.java:71)
at org.apache.solr.core.HdfsDirectoryFactory.createBlockCache(HdfsDirectoryFactory.java:309)
... 14 more
FAILED: org.apache.solr.search.TestRecoveryHdfs.testBufferingFlags
Error Message:
SolrCore 'collection1' is not available due to init failure: The max direct memory is likely too low. Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~8.388608 MB.
Stack Trace:
org.apache.solr.core.SolrCoreInitializationException: SolrCore 'collection1' is not available due to init failure: The max direct memory is likely too low. Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~8.388608 MB.
at __randomizedtesting.SeedInfo.seed([524737AC3F023EA7:D3BD56E1884A505F]:0)
at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1388)
at org.apache.solr.util.TestHarness.getCore(TestHarness.java:250)
at org.apache.solr.util.TestHarness$LocalRequestFactory.makeRequest(TestHarness.java:469)
at org.apache.solr.SolrTestCaseJ4.req(SolrTestCaseJ4.java:1187)
at org.apache.solr.search.TestRecoveryHdfs.testBufferingFlags(TestRecoveryHdfs.java:526)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.common.SolrException: The max direct memory is likely too low. Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~8.388608 MB.
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1002)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:855)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:979)
at org.apache.solr.core.CoreContainer.lambda$load$7(CoreContainer.java:611)
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
... 1 more
Caused by: java.lang.RuntimeException: The max direct memory is likely too low. Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~8.388608 MB.
at org.apache.solr.core.HdfsDirectoryFactory.createBlockCache(HdfsDirectoryFactory.java:311)
at org.apache.solr.core.HdfsDirectoryFactory.getBlockDirectoryCache(HdfsDirectoryFactory.java:287)
at org.apache.solr.core.HdfsDirectoryFactory.create(HdfsDirectoryFactory.java:227)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:346)
at org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:490)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:942)
... 9 more
Caused by: java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Bits.java:693)
at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
at org.apache.solr.store.blockcache.BlockCache.<init>(BlockCache.java:71)
at org.apache.solr.core.HdfsDirectoryFactory.createBlockCache(HdfsDirectoryFactory.java:309)
... 14 more
FAILED: org.apache.solr.search.TestRecoveryHdfs.testCorruptLog
Error Message:
SolrCore 'collection1' is not available due to init failure: The max direct memory is likely too low. Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~8.388608 MB.
Stack Trace:
org.apache.solr.core.SolrCoreInitializationException: SolrCore 'collection1' is not available due to init failure: The max direct memory is likely too low. Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~8.388608 MB.
at __randomizedtesting.SeedInfo.seed([524737AC3F023EA7:A09A837FC53A6F7B]:0)
at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1388)
at org.apache.solr.util.TestHarness.getCore(TestHarness.java:250)
at org.apache.solr.search.TestRecoveryHdfs.testCorruptLog(TestRecoveryHdfs.java:881)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.common.SolrException: The max direct memory is likely too low. Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~8.388608 MB.
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1002)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:855)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:979)
at org.apache.solr.core.CoreContainer.lambda$load$7(CoreContainer.java:611)
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
... 1 more
Caused by: java.lang.RuntimeException: The max direct memory is likely too low. Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~8.388608 MB.
at org.apache.solr.core.HdfsDirectoryFactory.createBlockCache(HdfsDirectoryFactory.java:311)
at org.apache.solr.core.HdfsDirectoryFactory.getBlockDirectoryCache(HdfsDirectoryFactory.java:287)
at org.apache.solr.core.HdfsDirectoryFactory.create(HdfsDirectoryFactory.java:227)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:346)
at org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:490)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:942)
... 9 more
Caused by: java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Bits.java:693)
at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
at org.apache.solr.store.blockcache.BlockCache.<init>(BlockCache.java:71)
at org.apache.solr.core.HdfsDirectoryFactory.createBlockCache(HdfsDirectoryFactory.java:309)
... 14 more
FAILED: org.apache.solr.search.TestRecoveryHdfs.testCleanShutdown
Error Message:
SolrCore 'collection1' is not available due to init failure: The max direct memory is likely too low. Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~8.388608 MB.
Stack Trace:
org.apache.solr.core.SolrCoreInitializationException: SolrCore 'collection1' is not available due to init failure: The max direct memory is likely too low. Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~8.388608 MB.
at __randomizedtesting.SeedInfo.seed([524737AC3F023EA7:CC088F6B5A9B66E2]:0)
at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1388)
at org.apache.solr.util.TestHarness.getCore(TestHarness.java:250)
at org.apache.solr.util.TestHarness$LocalRequestFactory.makeRequest(TestHarness.java:469)
at org.apache.solr.SolrTestCaseJ4.req(SolrTestCaseJ4.java:1187)
at org.apache.solr.search.TestRecoveryHdfs.testCleanShutdown(TestRecoveryHdfs.java:659)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.common.SolrException: The max direct memory is likely too low. Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~8.388608 MB.
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1002)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:855)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:979)
at org.apache.solr.core.CoreContainer.lambda$load$7(CoreContainer.java:611)
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
... 1 more
Caused by: java.lang.RuntimeException: The max direct memory is likely too low. Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~8.388608 MB.
at org.apache.solr.core.HdfsDirectoryFactory.createBlockCache(HdfsDirectoryFactory.java:311)
at org.apache.solr.core.HdfsDirectoryFactory.getBlockDirectoryCache(HdfsDirectoryFactory.java:287)
at org.apache.solr.core.HdfsDirectoryFactory.create(HdfsDirectoryFactory.java:227)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:346)
at org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:490)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:942)
... 9 more
Caused by: java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Bits.java:693)
at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
at org.apache.solr.store.blockcache.BlockCache.<init>(BlockCache.java:71)
at org.apache.solr.core.HdfsDirectoryFactory.createBlockCache(HdfsDirectoryFactory.java:309)
... 14 more
FAILED: org.apache.solr.search.TestRecoveryHdfs.testRemoveOldLogs
Error Message:
org.apache.solr.core.SolrCoreInitializationException: SolrCore 'collection1' is not available due to init failure: The max direct memory is likely too low. Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~8.388608 MB.
Stack Trace:
java.lang.RuntimeException: org.apache.solr.core.SolrCoreInitializationException: SolrCore 'collection1' is not available due to init failure: The max direct memory is likely too low. Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~8.388608 MB.
at __randomizedtesting.SeedInfo.seed([524737AC3F023EA7:73DBF607F738B75F]:0)
at org.apache.solr.SolrTestCaseJ4.clearIndex(SolrTestCaseJ4.java:1239)
at org.apache.solr.search.TestRecoveryHdfs.testRemoveOldLogs(TestRecoveryHdfs.java:716)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.core.SolrCoreInitializationException: SolrCore 'collection1' is not available due to init failure: The max direct memory is likely too low. Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~8.388608 MB.
at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1388)
at org.apache.solr.util.TestHarness.getCore(TestHarness.java:250)
at org.apache.solr.SolrTestCaseJ4.updateJ(SolrTestCaseJ4.java:1245)
at org.apache.solr.SolrTestCaseJ4.deleteByQueryAndGetVersion(SolrTestCaseJ4.java:1451)
at org.apache.solr.SolrTestCaseJ4.clearIndex(SolrTestCaseJ4.java:1236)
... 40 more
Caused by: org.apache.solr.common.SolrException: The max direct memory is likely too low. Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~8.388608 MB.
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1002)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:855)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:979)
at org.apache.solr.core.CoreContainer.lambda$load$7(CoreContainer.java:611)
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
... 1 more
Caused by: java.lang.RuntimeException: The max direct memory is likely too low. Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~8.388608 MB.
at org.apache.solr.core.HdfsDirectoryFactory.createBlockCache(HdfsDirectoryFactory.java:311)
at org.apache.solr.core.HdfsDirectoryFactory.getBlockDirectoryCache(HdfsDirectoryFactory.java:287)
at org.apache.solr.core.HdfsDirectoryFactory.create(HdfsDirectoryFactory.java:227)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:346)
at org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:490)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:942)
... 9 more
Caused by: java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Bits.java:693)
at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
at org.apache.solr.store.blockcache.BlockCache.<init>(BlockCache.java:71)
at org.apache.solr.core.HdfsDirectoryFactory.createBlockCache(HdfsDirectoryFactory.java:309)
... 14 more
Build Log:
[...truncated 11619 lines...]
[junit4] Suite: org.apache.solr.core.TestSolrConfigHandler
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.core.TestSolrConfigHandler_524737AC3F023EA7-001/init-core-data-001
[junit4] 2> 535069 WARN (SUITE-TestSolrConfigHandler-seed#[524737AC3F023EA7]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
[junit4] 2> 535096 INFO (SUITE-TestSolrConfigHandler-seed#[524737AC3F023EA7]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields
[junit4] 2> 535098 INFO (SUITE-TestSolrConfigHandler-seed#[524737AC3F023EA7]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 535112 INFO (TEST-TestSolrConfigHandler.testProperty-seed#[524737AC3F023EA7]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProperty
[junit4] 2> 535320 INFO (TEST-TestSolrConfigHandler.testProperty-seed#[524737AC3F023EA7]) [ ] o.a.s.SolrTestCaseJ4 ####initCore
[junit4] 2> 535320 INFO (TEST-TestSolrConfigHandler.testProperty-seed#[524737AC3F023EA7]) [ ] o.a.s.SolrTestCaseJ4 ####initCore end
[junit4] 2> 535321 INFO (TEST-TestSolrConfigHandler.testProperty-seed#[524737AC3F023EA7]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.core.TestSolrConfigHandler_524737AC3F023EA7-001/tempDir-003/cores/core
[junit4] 2> 535323 INFO (TEST-TestSolrConfigHandler.testProperty-seed#[524737AC3F023EA7]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 535324 INFO (TEST-TestSolrConfigHandler.testProperty-seed#[524737AC3F023EA7]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@307eabb4{/solr,null,AVAILABLE}
[junit4] 2> 535325 INFO (TEST-TestSolrConfigHandler.testProperty-seed#[524737AC3F023EA7]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2494f8bf{HTTP/1.1,[http/1.1]}{127.0.0.1:46995}
[junit4] 2> 535325 INFO (TEST-TestSolrConfigHandler.testProperty-seed#[524737AC3F023EA7]) [ ] o.e.j.s.Server Started @538279ms
[junit4] 2> 535325 INFO (TEST-TestSolrConfigHandler.testProperty-seed#[524737AC3F023EA7]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=46995, configSetBaseDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.core.TestSolrConfigHandler_524737AC3F023EA7-001/tempDir-001, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.core.TestSolrConfigHandler_524737AC3F023EA7-001/tempDir-003/cores}
[junit4] 2> 535326 ERROR (TEST-TestSolrConfigHandler.testProperty-seed#[524737AC3F023EA7]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 535326 INFO (TEST-TestSolrConfigHandler.testProperty-seed#[524737AC3F023EA7]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 535326 INFO (TEST-TestSolrConfigHandler.testProperty-seed#[524737AC3F023EA7]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 535326 INFO (TEST-TestSolrConfigHandler.testProperty-seed#[524737AC3F023EA7]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 535326 INFO (TEST-TestSolrConfigHandler.testProperty-seed#[524737AC3F023EA7]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-06-13T18:47:42.073Z
[junit4] 2> 535326 INFO (TEST-TestSolrConfigHandler.testProperty-seed#[524737AC3F023EA7]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.core.TestSolrConfigHandler_524737AC3F023EA7-001/tempDir-001/solr.xml
[junit4] 2> 535338 INFO (TEST-TestSolrConfigHandler.testProperty-seed#[524737AC3F023EA7]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@ea5390c, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 535378 INFO (TEST-TestSolrConfigHandler.testProperty-seed#[524737AC3F023EA7]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@ea5390c
[junit4] 2> 535383 INFO (TEST-TestSolrConfigHandler.testProperty-seed#[524737AC3F023EA7]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@ea5390c
[junit4] 2> 535384 INFO (TEST-TestSolrConfigHandler.testProperty-seed#[524737AC3F023EA7]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@ea5390c
[junit4] 2> 535432 INFO (TEST-TestSolrConfigHandler.testProperty-seed#[524737AC3F023EA7]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.core.TestSolrConfigHandler_524737AC3F023EA7-001/tempDir-003/cores
[junit4] 2> 535432 INFO (TEST-TestSolrConfigHandler.testProperty-seed#[524737AC3F023EA7]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 535446 INFO (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.core.TestSolrConfigHandler_524737AC3F023EA7-001/tempDir-001/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.core.TestSolrConfigHandler_524737AC3F023EA7-001/tempDir-001/collection1/lib/classes]
[junit4] 2> 535716 INFO (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 535725 INFO (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.s.ManagedIndexSchemaFactory The schema is configured as managed, but managed schema resource managed-schema not found - loading non-managed schema schema-rest.xml instead
[junit4] 2> 535729 INFO (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test-rest
[junit4] 2> 535946 WARN (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.s.IndexSchema Field title is not multivalued and destination for multiple copyFields (3)
[junit4] 2> 535948 INFO (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.r.RestManager Registered ManagedResource impl org.apache.solr.rest.schema.analysis.ManagedWordSetResource for path /schema/analysis/stopwords/german
[junit4] 2> 535948 INFO (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.r.RestManager Registered ManagedResource impl org.apache.solr.rest.schema.analysis.ManagedSynonymFilterFactory$SynonymManager for path /schema/analysis/synonyms/german
[junit4] 2> 535949 INFO (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.r.RestManager Registered ManagedResource impl org.apache.solr.rest.schema.analysis.ManagedWordSetResource for path /schema/analysis/stopwords/english
[junit4] 2> 535949 INFO (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.r.RestManager Registered ManagedResource impl org.apache.solr.rest.schema.analysis.ManagedSynonymGraphFilterFactory$SynonymManager for path /schema/analysis/synonyms/englishgraph
[junit4] 2> 535949 INFO (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.r.RestManager Added observer of type org.apache.solr.rest.schema.analysis.ManagedStopFilterFactory to existing ManagedResource /schema/analysis/stopwords/english
[junit4] 2> 535949 INFO (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.r.RestManager Added observer of type org.apache.solr.rest.schema.analysis.ManagedSynonymGraphFilterFactory to existing ManagedResource /schema/analysis/synonyms/englishgraph
[junit4] 2> 535950 INFO (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.r.RestManager Added observer of type org.apache.solr.rest.schema.analysis.ManagedStopFilterFactory to existing ManagedResource /schema/analysis/stopwords/german
[junit4] 2> 535950 INFO (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.r.RestManager Registered ManagedResource impl org.apache.solr.rest.schema.analysis.ManagedSynonymGraphFilterFactory$SynonymManager for path /schema/analysis/synonyms/germangraph
[junit4] 2> 535950 INFO (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.r.RestManager Added observer of type org.apache.solr.rest.schema.analysis.ManagedStopFilterFactory to existing ManagedResource /schema/analysis/stopwords/german
[junit4] 2> 535950 INFO (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.r.RestManager Added observer of type org.apache.solr.rest.schema.analysis.ManagedSynonymGraphFilterFactory to existing ManagedResource /schema/analysis/synonyms/germangraph
[junit4] 2> 535954 INFO (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.r.RestManager Added observer of type org.apache.solr.rest.schema.analysis.ManagedStopFilterFactory to existing ManagedResource /schema/analysis/stopwords/english
[junit4] 2> 535954 INFO (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.r.RestManager Registered ManagedResource impl org.apache.solr.rest.schema.analysis.ManagedSynonymFilterFactory$SynonymManager for path /schema/analysis/synonyms/english
[junit4] 2> 535954 INFO (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test-rest/1.6 with uniqueid field id
[junit4] 2> 535960 INFO (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.r.RestManager Added observer of type org.apache.solr.rest.schema.analysis.ManagedStopFilterFactory to existing ManagedResource /schema/analysis/stopwords/english
[junit4] 2> 535960 INFO (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.r.RestManager Added observer of type org.apache.solr.rest.schema.analysis.ManagedSynonymFilterFactory to existing ManagedResource /schema/analysis/synonyms/english
[junit4] 2> 535960 INFO (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.r.RestManager Added observer of type org.apache.solr.rest.schema.analysis.ManagedStopFilterFactory to existing ManagedResource /schema/analysis/stopwords/german
[junit4] 2> 535960 INFO (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.r.RestManager Added observer of type org.apache.solr.rest.schema.analysis.ManagedSynonymFilterFactory to existing ManagedResource /schema/analysis/synonyms/german
[junit4] 2> 535960 INFO (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.r.RestManager Added observer of type org.apache.solr.rest.schema.analysis.ManagedStopFilterFactory to existing ManagedResource /schema/analysis/stopwords/english
[junit4] 2> 535960 INFO (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.r.RestManager Added observer of type org.apache.solr.rest.schema.analysis.ManagedSynonymGraphFilterFactory to existing ManagedResource /schema/analysis/synonyms/englishgraph
[junit4] 2> 535960 INFO (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.r.RestManager Added observer of type org.apache.solr.rest.schema.analysis.ManagedStopFilterFactory to existing ManagedResource /schema/analysis/stopwords/english
[junit4] 2> 535960 INFO (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.r.RestManager Added observer of type org.apache.solr.rest.schema.analysis.ManagedSynonymGraphFilterFactory to existing ManagedResource /schema/analysis/synonyms/englishgraph
[junit4] 2> 535960 INFO (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.r.RestManager Added observer of type org.apache.solr.rest.schema.analysis.ManagedStopFilterFactory to existing ManagedResource /schema/analysis/stopwords/german
[junit4] 2> 535960 INFO (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.r.RestManager Added observer of type org.apache.solr.rest.schema.analysis.ManagedSynonymGraphFilterFactory to existing ManagedResource /schema/analysis/synonyms/germangraph
[junit4] 2> 535960 INFO (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.r.RestManager Added observer of type org.apache.solr.rest.schema.analysis.ManagedStopFilterFactory to existing ManagedResource /schema/analysis/stopwords/german
[junit4] 2> 535960 INFO (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.r.RestManager Added observer of type org.apache.solr.rest.schema.analysis.ManagedSynonymGraphFilterFactory to existing ManagedResource /schema/analysis/synonyms/germangraph
[junit4] 2> 535962 INFO (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.s.ManagedIndexSchema Upgraded to managed schema at /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.core.TestSolrConfigHandler_524737AC3F023EA7-001/tempDir-001/collection1/conf/managed-schema
[junit4] 2> 535994 INFO (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.s.ManagedIndexSchemaFactory After upgrading to managed schema, renamed the non-managed schema /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.core.TestSolrConfigHandler_524737AC3F023EA7-001/tempDir-001/collection1/conf/schema-rest.xml to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.core.TestSolrConfigHandler_524737AC3F023EA7-001/tempDir-001/collection1/conf/schema-rest.xml.bak
[junit4] 2> 535995 INFO (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from configset /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.core.TestSolrConfigHandler_524737AC3F023EA7-001/tempDir-001/collection1, trusted=true
[junit4] 2> 535997 INFO (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@ea5390c
[junit4] 2> 535997 INFO (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 535997 INFO (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.core.TestSolrConfigHandler_524737AC3F023EA7-001/tempDir-001/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.core.TestSolrConfigHandler_524737AC3F023EA7-001/tempDir-003/cores/core/data/]
[junit4] 2> 536002 INFO (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=29, maxMergeAtOnceExplicit=44, maxMergedSegmentMB=67.228515625, floorSegmentMB=1.5390625, forceMergeDeletesPctAllowed=8.783959569640189, segmentsPerTier=46.0, maxCFSSegmentSizeMB=1.236328125, noCFSRatio=0.43519450857273345
[junit4] 2> 536012 WARN (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, registerPath=/solr,/v2, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 536082 INFO (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 536082 INFO (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 536083 INFO (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=15, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4684857531786689]
[junit4] 2> 536085 INFO (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5b8a7323[collection1] main]
[junit4] 2> 536085 INFO (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.core.TestSolrConfigHandler_524737AC3F023EA7-001/tempDir-001/collection1/conf
[junit4] 2> 536085 WARN (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.r.ManagedResource No stored data found for /schema/analysis/stopwords/english
[junit4] 2> 536086 INFO (coreLoadExecutor-6515-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage Saved JSON object to path _schema_analysis_stopwords_english.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.core.TestSolrConfigHandler_524737AC3F023EA7-001/tempDir-001/collection1/conf
[junit4] 2> 536086 INFO (coreLoadExecutor-6515-t
[...truncated too long message...]
ns/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test/org/apache/solr/response/transform/TestSubQueryTransformer.java (at line 387)
[ecj-lint] unmarshalled = (NamedList<Object>) new JavaBinCodec().unmarshal(
[ecj-lint] ^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 22. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test/org/apache/solr/search/TestDocSet.java (at line 242)
[ecj-lint] return loadfactor!=0 ? new HashDocSet(a,0,n,1/loadfactor) : new HashDocSet(a,0,n);
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
[ecj-lint] ----------
[ecj-lint] 23. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test/org/apache/solr/search/TestDocSet.java (at line 527)
[ecj-lint] DocSet a = new BitDocSet(bs);
[ecj-lint] ^
[ecj-lint] Resource leak: 'a' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 24. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test/org/apache/solr/security/TestPKIAuthenticationPlugin.java (at line 76)
[ecj-lint] final MockPKIAuthenticationPlugin mock = new MockPKIAuthenticationPlugin(null, nodeName);
[ecj-lint] ^^^^
[ecj-lint] Resource leak: 'mock' is never closed
[ecj-lint] ----------
[ecj-lint] 25. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test/org/apache/solr/security/TestPKIAuthenticationPlugin.java (at line 131)
[ecj-lint] MockPKIAuthenticationPlugin mock1 = new MockPKIAuthenticationPlugin(null, nodeName) {
[ecj-lint] ^^^^^
[ecj-lint] Resource leak: 'mock1' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 26. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test/org/apache/solr/security/TestRuleBasedAuthorizationPlugin.java (at line 380)
[ecj-lint] RuleBasedAuthorizationPlugin plugin = new RuleBasedAuthorizationPlugin();
[ecj-lint] ^^^^^^
[ecj-lint] Resource leak: 'plugin' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 27. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test/org/apache/solr/security/TestSha256AuthenticationProvider.java (at line 49)
[ecj-lint] BasicAuthPlugin basicAuthPlugin = new BasicAuthPlugin();
[ecj-lint] ^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: 'basicAuthPlugin' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 28. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test/org/apache/solr/spelling/SimpleQueryConverter.java (at line 42)
[ecj-lint] WhitespaceAnalyzer analyzer = new WhitespaceAnalyzer();
[ecj-lint] ^^^^^^^^
[ecj-lint] Resource leak: 'analyzer' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 29. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test/org/apache/solr/uninverting/TestFieldCacheVsDocValues.java (at line 139)
[ecj-lint] IndexWriter w = new IndexWriter(d, newIndexWriterConfig(analyzer));
[ecj-lint] ^
[ecj-lint] Resource leak: 'w' is never closed
[ecj-lint] ----------
[ecj-lint] 30. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test/org/apache/solr/uninverting/TestFieldCacheVsDocValues.java (at line 172)
[ecj-lint] throw iae;
[ecj-lint] ^^^^^^^^^^
[ecj-lint] Resource leak: 'w' is not closed at this location
[ecj-lint] ----------
[ecj-lint] 31. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test/org/apache/solr/uninverting/TestFieldCacheVsDocValues.java (at line 178)
[ecj-lint] return;
[ecj-lint] ^^^^^^^
[ecj-lint] Resource leak: 'w' is not closed at this location
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 32. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test/org/apache/solr/update/AddBlockUpdateTest.java (at line 415)
[ecj-lint] new JavaBinCodec().marshal(topDocument, os);
[ecj-lint] ^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
[ecj-lint] ----------
[ecj-lint] 33. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test/org/apache/solr/update/AddBlockUpdateTest.java (at line 419)
[ecj-lint] SolrInputDocument result = (SolrInputDocument) new JavaBinCodec().unmarshal(is);
[ecj-lint] ^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 34. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test/org/apache/solr/update/SolrCmdDistributorTest.java (at line 134)
[ecj-lint] SolrCmdDistributor cmdDistrib = new SolrCmdDistributor(updateShardHandler);
[ecj-lint] ^^^^^^^^^^
[ecj-lint] Resource leak: 'cmdDistrib' is never closed
[ecj-lint] ----------
[ecj-lint] 35. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test/org/apache/solr/update/SolrCmdDistributorTest.java (at line 333)
[ecj-lint] SolrCmdDistributor cmdDistrib = new SolrCmdDistributor(streamingClients, 5, 0);
[ecj-lint] ^^^^^^^^^^
[ecj-lint] Resource leak: 'cmdDistrib' is never closed
[ecj-lint] ----------
[ecj-lint] 36. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test/org/apache/solr/update/SolrCmdDistributorTest.java (at line 367)
[ecj-lint] SolrCmdDistributor cmdDistrib = new SolrCmdDistributor(streamingClients, 5, 0);
[ecj-lint] ^^^^^^^^^^
[ecj-lint] Resource leak: 'cmdDistrib' is never closed
[ecj-lint] ----------
[ecj-lint] 37. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test/org/apache/solr/update/SolrCmdDistributorTest.java (at line 413)
[ecj-lint] SolrCmdDistributor cmdDistrib = new SolrCmdDistributor(streamingClients, 5, 0);
[ecj-lint] ^^^^^^^^^^
[ecj-lint] Resource leak: 'cmdDistrib' is never closed
[ecj-lint] ----------
[ecj-lint] 38. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test/org/apache/solr/update/SolrCmdDistributorTest.java (at line 458)
[ecj-lint] SolrCmdDistributor cmdDistrib = new SolrCmdDistributor(updateShardHandler);
[ecj-lint] ^^^^^^^^^^
[ecj-lint] Resource leak: 'cmdDistrib' is never closed
[ecj-lint] ----------
[ecj-lint] 39. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test/org/apache/solr/update/SolrCmdDistributorTest.java (at line 516)
[ecj-lint] SolrCmdDistributor cmdDistrib = new SolrCmdDistributor(updateShardHandler);
[ecj-lint] ^^^^^^^^^^
[ecj-lint] Resource leak: 'cmdDistrib' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 40. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test/org/apache/solr/update/SolrIndexSplitterTest.java (at line 181)
[ecj-lint] EmbeddedSolrServer server1 = new EmbeddedSolrServer(h.getCoreContainer(), "split1");
[ecj-lint] ^^^^^^^
[ecj-lint] Resource leak: 'server1' is never closed
[ecj-lint] ----------
[ecj-lint] 41. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test/org/apache/solr/update/SolrIndexSplitterTest.java (at line 182)
[ecj-lint] EmbeddedSolrServer server2 = new EmbeddedSolrServer(h.getCoreContainer(), "split2");
[ecj-lint] ^^^^^^^
[ecj-lint] Resource leak: 'server2' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 42. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test/org/apache/solr/update/processor/RecordingUpdateProcessorFactory.java (at line 67)
[ecj-lint] return recording ? new RecordingUpdateRequestProcessor(commandQueue, next) : next;
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
[ecj-lint] ----------
[ecj-lint] 42 problems (3 errors, 39 warnings)
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/build.xml:783: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/build.xml:101: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build.xml:689: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/common-build.xml:2010: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/common-build.xml:2043: Compile failed; see the compiler error output for details.
Total time: 290 minutes 16 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1323 - Failure
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1323/
2 tests failed.
FAILED: org.apache.lucene.search.TestInetAddressRangeQueries.testRandomBig
Error Message:
Java heap space
Stack Trace:
java.lang.OutOfMemoryError: Java heap space
at __randomizedtesting.SeedInfo.seed([6AE5E766A0153ED9:EDB29AE9314C4259]:0)
at org.apache.lucene.util.bkd.HeapPointWriter.writePackedValue(HeapPointWriter.java:107)
at org.apache.lucene.util.bkd.HeapPointWriter.append(HeapPointWriter.java:128)
at org.apache.lucene.util.bkd.PointReader.split(PointReader.java:74)
at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1791)
at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1805)
at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1805)
at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1805)
at org.apache.lucene.util.bkd.BKDWriter.finish(BKDWriter.java:1008)
at org.apache.lucene.codecs.lucene60.Lucene60PointsWriter.writeField(Lucene60PointsWriter.java:130)
at org.apache.lucene.codecs.PointsWriter.mergeOneField(PointsWriter.java:62)
at org.apache.lucene.codecs.lucene60.Lucene60PointsWriter.merge(Lucene60PointsWriter.java:223)
at org.apache.lucene.index.SegmentMerger.mergePoints(SegmentMerger.java:187)
at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:136)
at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4361)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3933)
at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2082)
at org.apache.lucene.index.IndexWriter.doAfterSegmentFlushed(IndexWriter.java:4992)
at org.apache.lucene.index.DocumentsWriter$MergePendingEvent.process(DocumentsWriter.java:731)
at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5030)
at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5021)
at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1573)
at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1315)
at org.apache.lucene.search.BaseRangeFieldQueryTestCase.verify(BaseRangeFieldQueryTestCase.java:190)
at org.apache.lucene.search.BaseRangeFieldQueryTestCase.doTestRandom(BaseRangeFieldQueryTestCase.java:160)
at org.apache.lucene.search.BaseRangeFieldQueryTestCase.testRandomBig(BaseRangeFieldQueryTestCase.java:75)
at org.apache.lucene.search.TestInetAddressRangeQueries.testRandomBig(TestInetAddressRangeQueries.java:81)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
FAILED: org.apache.solr.TestDistributedSearch.test
Error Message:
Captured an uncaught exception in thread: Thread[id=44906, name=Thread-33143, state=RUNNABLE, group=TGRP-TestDistributedSearch]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=44906, name=Thread-33143, state=RUNNABLE, group=TGRP-TestDistributedSearch]
at __randomizedtesting.SeedInfo.seed([96F460804B43D487:1EA05F5AE5BFB97F]:0)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:39331//collection1: java.lang.NullPointerException
at org.apache.solr.search.grouping.distributed.responseprocessor.StoredFieldsShardResponseProcessor.process(StoredFieldsShardResponseProcessor.java:41)
at org.apache.solr.handler.component.QueryComponent.handleGroupedResponses(QueryComponent.java:768)
at org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:751)
at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:423)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:178)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2487)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:722)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:528)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:374)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:318)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:462)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:534)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([96F460804B43D487]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:610)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:237)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:957)
at org.apache.solr.TestDistributedSearch$1.run(TestDistributedSearch.java:1140)
Build Log:
[...truncated 8331 lines...]
[junit4] Suite: org.apache.lucene.search.TestInetAddressRangeQueries
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestInetAddressRangeQueries -Dtests.method=testRandomBig -Dtests.seed=6AE5E766A0153ED9 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=de-DE -Dtests.timezone=America/Cayman -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] ERROR 122s J1 | TestInetAddressRangeQueries.testRandomBig <<<
[junit4] > Throwable #1: java.lang.OutOfMemoryError: Java heap space
[junit4] > at __randomizedtesting.SeedInfo.seed([6AE5E766A0153ED9:EDB29AE9314C4259]:0)
[junit4] > at org.apache.lucene.util.bkd.HeapPointWriter.writePackedValue(HeapPointWriter.java:107)
[junit4] > at org.apache.lucene.util.bkd.HeapPointWriter.append(HeapPointWriter.java:128)
[junit4] > at org.apache.lucene.util.bkd.PointReader.split(PointReader.java:74)
[junit4] > at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1791)
[junit4] > at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1805)
[junit4] > at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1805)
[junit4] > at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1805)
[junit4] > at org.apache.lucene.util.bkd.BKDWriter.finish(BKDWriter.java:1008)
[junit4] > at org.apache.lucene.codecs.lucene60.Lucene60PointsWriter.writeField(Lucene60PointsWriter.java:130)
[junit4] > at org.apache.lucene.codecs.PointsWriter.mergeOneField(PointsWriter.java:62)
[junit4] > at org.apache.lucene.codecs.lucene60.Lucene60PointsWriter.merge(Lucene60PointsWriter.java:223)
[junit4] > at org.apache.lucene.index.SegmentMerger.mergePoints(SegmentMerger.java:187)
[junit4] > at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:136)
[junit4] > at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4361)
[junit4] > at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3933)
[junit4] > at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
[junit4] > at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2082)
[junit4] > at org.apache.lucene.index.IndexWriter.doAfterSegmentFlushed(IndexWriter.java:4992)
[junit4] > at org.apache.lucene.index.DocumentsWriter$MergePendingEvent.process(DocumentsWriter.java:731)
[junit4] > at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5030)
[junit4] > at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5021)
[junit4] > at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1573)
[junit4] > at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1315)
[junit4] > at org.apache.lucene.search.BaseRangeFieldQueryTestCase.verify(BaseRangeFieldQueryTestCase.java:190)
[junit4] > at org.apache.lucene.search.BaseRangeFieldQueryTestCase.doTestRandom(BaseRangeFieldQueryTestCase.java:160)
[junit4] > at org.apache.lucene.search.BaseRangeFieldQueryTestCase.testRandomBig(BaseRangeFieldQueryTestCase.java:75)
[junit4] > at org.apache.lucene.search.TestInetAddressRangeQueries.testRandomBig(TestInetAddressRangeQueries.java:81)
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/build/misc/test/J1/temp/lucene.search.TestInetAddressRangeQueries_6AE5E766A0153ED9-001
[junit4] 2> NOTE: test params are: codec=DummyCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=DUMMY, chunkSize=25137, maxDocsPerChunk=1, blockSize=4), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=DUMMY, chunkSize=25137, blockSize=4)), sim=RandomSimilarity(queryNorm=false): {}, locale=de-DE, timezone=America/Cayman
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_131 (64-bit)/cpus=4,threads=1,free=271326872,total=317718528
[junit4] 2> NOTE: All tests run in this JVM: [TestLazyDocument, TestHighFreqTerms, TestIndexSplitter, TestPKIndexSplitter, TestDocValuesStatsCollector, TestInetAddressRangeQueries]
[junit4] Completed [13/13 (1!)] on J1 in 136.64s, 4 tests, 1 error <<< FAILURES!
[...truncated 1 lines...]
[junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/build/misc/test/temp/junit4-J1-20170612_174903_9697726839819003088903.sysout
[junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
[junit4] java.lang.OutOfMemoryError: Java heap space
[junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/heapdumps/java_pid30482.hprof ...
[junit4] Heap dump file created [571547327 bytes in 3.816 secs]
[junit4] <<< JVM J1: EOF ----
[...truncated 4500 lines...]
[junit4] Suite: org.apache.solr.TestDistributedSearch
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_96F460804B43D487-001/init-core-data-001
[junit4] 2> 2562074 WARN (SUITE-TestDistributedSearch-seed#[96F460804B43D487]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=32 numCloses=32
[junit4] 2> 2562075 INFO (SUITE-TestDistributedSearch-seed#[96F460804B43D487]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields
[junit4] 2> 2562076 INFO (SUITE-TestDistributedSearch-seed#[96F460804B43D487]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-9061)
[junit4] 2> 2562076 INFO (SUITE-TestDistributedSearch-seed#[96F460804B43D487]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 2> 2562188 INFO (TEST-TestDistributedSearch.test-seed#[96F460804B43D487]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_96F460804B43D487-001/tempDir-001/control/cores/collection1
[junit4] 2> 2562190 INFO (TEST-TestDistributedSearch.test-seed#[96F460804B43D487]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 2562191 INFO (TEST-TestDistributedSearch.test-seed#[96F460804B43D487]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2e006083{/,null,AVAILABLE}
[junit4] 2> 2562198 INFO (TEST-TestDistributedSearch.test-seed#[96F460804B43D487]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3e75fd7a{HTTP/1.1,[http/1.1]}{127.0.0.1:34743}
[junit4] 2> 2562198 INFO (TEST-TestDistributedSearch.test-seed#[96F460804B43D487]) [ ] o.e.j.s.Server Started @2565225ms
[junit4] 2> 2562198 INFO (TEST-TestDistributedSearch.test-seed#[96F460804B43D487]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, hostPort=34743, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_96F460804B43D487-001/tempDir-001/control/cores}
[junit4] 2> 2562199 ERROR (TEST-TestDistributedSearch.test-seed#[96F460804B43D487]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2562199 INFO (TEST-TestDistributedSearch.test-seed#[96F460804B43D487]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 2562199 INFO (TEST-TestDistributedSearch.test-seed#[96F460804B43D487]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 2562199 INFO (TEST-TestDistributedSearch.test-seed#[96F460804B43D487]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2562199 INFO (TEST-TestDistributedSearch.test-seed#[96F460804B43D487]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-06-12T19:44:29.713Z
[junit4] 2> 2562199 INFO (TEST-TestDistributedSearch.test-seed#[96F460804B43D487]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_96F460804B43D487-001/tempDir-001/control/solr.xml
[junit4] 2> 2562218 INFO (TEST-TestDistributedSearch.test-seed#[96F460804B43D487]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@7d23d27c, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 2562263 INFO (TEST-TestDistributedSearch.test-seed#[96F460804B43D487]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7d23d27c
[junit4] 2> 2562268 INFO (TEST-TestDistributedSearch.test-seed#[96F460804B43D487]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7d23d27c
[junit4] 2> 2562268 INFO (TEST-TestDistributedSearch.test-seed#[96F460804B43D487]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7d23d27c
[junit4] 2> 2562270 INFO (TEST-TestDistributedSearch.test-seed#[96F460804B43D487]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_96F460804B43D487-001/tempDir-001/control/cores
[junit4] 2> 2562270 INFO (TEST-TestDistributedSearch.test-seed#[96F460804B43D487]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 2562271 INFO (coreLoadExecutor-3750-thread-1) [ x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
[junit4] 2> 2562313 INFO (coreLoadExecutor-3750-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2562337 INFO (coreLoadExecutor-3750-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 2562440 INFO (coreLoadExecutor-3750-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 2562445 INFO (coreLoadExecutor-3750-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from configset /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1, trusted=true
[junit4] 2> 2562445 INFO (coreLoadExecutor-3750-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7d23d27c
[junit4] 2> 2562445 INFO (coreLoadExecutor-3750-thread-1) [ x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2562446 INFO (coreLoadExecutor-3750-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_96F460804B43D487-001/tempDir-001/control/cores/collection1/data/]
[junit4] 2> 2562447 INFO (coreLoadExecutor-3750-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=7, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 2562505 INFO (coreLoadExecutor-3750-thread-1) [ x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2562505 INFO (coreLoadExecutor-3750-thread-1) [ x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2562506 INFO (coreLoadExecutor-3750-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2562506 INFO (coreLoadExecutor-3750-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2562507 INFO (coreLoadExecutor-3750-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=27, maxMergeAtOnceExplicit=35, maxMergedSegmentMB=60.5263671875, floorSegmentMB=1.130859375, forceMergeDeletesPctAllowed=1.8812980484576602, segmentsPerTier=23.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 2562508 INFO (coreLoadExecutor-3750-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@47029ba6[collection1] main]
[junit4] 2> 2562508 INFO (coreLoadExecutor-3750-thread-1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
[junit4] 2> 2562517 INFO (coreLoadExecutor-3750-thread-1) [ x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4] 2> 2562543 INFO (coreLoadExecutor-3750-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2562543 INFO (searcherExecutor-3751-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
[junit4] 2> 2562543 INFO (coreLoadExecutor-3750-thread-1) [ x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1570029353101688832
[junit4] 2> 2562544 INFO (searcherExecutor-3751-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
[junit4] 2> 2562544 INFO (searcherExecutor-3751-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
[junit4] 2> 2562544 INFO (searcherExecutor-3751-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
[junit4] 2> 2562544 INFO (searcherExecutor-3751-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
[junit4] 2> 2562544 INFO (searcherExecutor-3751-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
[junit4] 2> 2562544 INFO (searcherExecutor-3751-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
[junit4] 2> 2562544 INFO (searcherExecutor-3751-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
[junit4] 2> 2562545 INFO (searcherExecutor-3751-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
[junit4] 2> 2562546 INFO (searcherExecutor-3751-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@47029ba6[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2562907 INFO (TEST-TestDistributedSearch.test-seed#[96F460804B43D487]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_96F460804B43D487-001/tempDir-001/shard0/cores/collection1
[junit4] 2> 2562909 INFO (TEST-TestDistributedSearch.test-seed#[96F460804B43D487]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 2562913 INFO (TEST-TestDistributedSearch.test-seed#[96F460804B43D487]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@114d3652{/,null,AVAILABLE}
[junit4] 2> 2562913 INFO (TEST-TestDistributedSearch.test-seed#[96F460804B43D487]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3a4c461d{HTTP/1.1,[http/1.1]}{127.0.0.1:60942}
[junit4] 2> 2562914 INFO (TEST-TestDistributedSearch.test-seed#[96F460804B43D487]) [ ] o.e.j.s.Server Started @2565941ms
[junit4] 2> 2562914 INFO (TEST-TestDistributedSearch.test-seed#[96F460804B43D487]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, hostPort=60942, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_96F460804B43D487-001/tempDir-001/shard0/cores}
[junit4] 2> 2562914 ERROR (TEST-TestDistributedSearch.test-seed#[96F460804B43D487]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2562914 INFO (TEST-TestDistributedSearch.test-seed#[96F460804B43D487]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 2562915 INFO (TEST-TestDistributedSearch.test-seed#[96F460804B43D487]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 2562915 INFO (TEST-TestDistributedSearch.test-seed#[96F460804B43D487]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2562915 INFO (TEST-TestDistributedSearch.test-seed#[96F460804B43D487]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-06-12T19:44:30.429Z
[junit4] 2> 2562915 INFO (TEST-TestDistributedSearch.test-seed#[96F460804B43D487]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_96F460804B43D487-001/tempDir-001/shard0/solr.xml
[junit4] 2> 2562920 INFO (TEST-TestDistributedSearch.test-seed#[96F460804B43D487]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@7d23d27c, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 2562955 INFO (TEST-TestDistributedSearch.test-seed#[96F460804B43D487]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7d23d27c
[junit4] 2> 2562961 INFO (TEST-TestDistributedSearch.test-seed#[96F460804B43D487]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7d23d27c
[junit4] 2> 2562961 INFO (TEST-TestDistributedSearch.test-seed#[96F460804B43D487]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7d23d27c
[junit4] 2> 2562962 INFO (TEST-TestDistributedSearch.test-seed#[96F460804B43D487]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_96F460804B43D487-001/tempDir-001/shard0/cores
[junit4] 2> 2562963 INFO (TEST-TestDistributedSearch.test-seed#[96F460804B43D487]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 2562963 INFO (coreLoadExecutor-3761-thread-1) [ x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
[junit4] 2> 2562995 INFO (coreLoadExecutor-3761-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2563018 INFO (coreLoadExecutor-3761-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 2563120 INFO (coreLoadExecutor-3761-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 2563125 INFO (coreLoadExecutor-3761-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from configset /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1, trusted=true
[junit4] 2> 2563126 INFO (coreLoadExecutor-3761-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7d23d27c
[junit4] 2> 2563126 INFO (coreLoadExecutor-3761-thread-1) [ x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2563126 INFO (coreLoadExecutor-3761-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_96F460804B43D487-001/tempDir-001/shard0/cores/collection1/data/]
[junit4] 2> 2563127 INFO (coreLoadExecutor-3761-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=7, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 2563188 INFO (coreLoadExecutor-3761-thread-1) [ x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2563188 INFO (coreLoadExecutor-3761-thread-1) [ x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2563189 INFO (coreLoadExecutor-3761-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2563189 INFO (coreLoadExecutor-3761-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2563190 INFO (coreLoadExecutor-3761-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=27, maxMergeAtOnceExplicit=35, maxMergedSegmentMB=60.5263671875, floorSegmentMB=1.130859375, forceMergeDeletesPctAllowed=1.8812980484576602, segmentsPerTier=23.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 2563191 INFO (coreLoadExecutor-3761-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1e850bca[collection1] main]
[junit4] 2> 2563191 INFO (coreLoadExecutor-3761-thread-1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
[junit4] 2> 2563202 INFO (coreLoadExecutor-3761-thread-1) [ x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4] 2> 2563228 INFO (coreLoadExecutor-3761-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2563228 INFO (searcherExecutor-3762-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
[junit4] 2> 2563228 INFO (coreLoadExecutor-3761-thread-1) [ x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1570029353819963392
[junit4] 2> 2563228 INFO (searcherExecutor-3762-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
[junit4] 2> 2563228 INFO (searcherExecutor-3762-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
[junit4] 2> 2563228 INFO (searcherExecutor-3762-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
[junit4] 2> 2563229 INFO (searcherExecutor-3762-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
[junit4] 2> 2563229 INFO (searcherExecutor-3762-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
[junit4] 2> 2563229 INFO (searcherExecutor-3762-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
[junit4] 2> 2563229 INFO (searcherExecutor-3762-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
[junit4] 2> 2563230 INFO (searcherExecutor-3762-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
[junit4] 2> 2563231 INFO (searcherExecutor-3762-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1e850bca[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2563488 INFO (TEST-TestDistributedSearch.test-seed#[96F460804B43D487]) [ ] o.a.s.SolrTestCaseJ4 ###Starting test
[junit4] 2> 2563491 INFO (qtp494089468-13184) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{deleteByQuery=*:* (-1570029354093641728)} 0 1
[junit4] 2> 2563493 INFO (qtp1207983899-13201) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{deleteByQuery=*:* (-1570029354096787456)} 0 1
[junit4] 2> 2563495 INFO (qtp494089468-13185) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[1 (1570029354098884608)]} 0 1
[junit4] 2> 2563497 INFO (qtp1207983899-13202) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[1 (1570029354100981760)]} 0 1
[junit4] 2> 2563498 INFO (qtp494089468-13186) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[2 (1570029354102030336)]} 0 0
[junit4] 2> 2563499 INFO (qtp1207983899-13203) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[2 (1570029354104127488)]} 0 0
[junit4] 2> 2563501 INFO (qtp494089468-13187) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[3 (1570029354105176064)]} 0 1
[junit4] 2> 2563502 INFO (qtp1207983899-13204) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[3 (1570029354107273216)]} 0 0
[junit4] 2> 2563503 INFO (qtp494089468-13188) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[4 (1570029354108321792)]} 0 0
[junit4] 2> 2563504 INFO (qtp1207983899-13204) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[4 (1570029354109370368)]} 0 0
[junit4] 2> 2563505 INFO (qtp494089468-13181) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[5 (1570029354110418944)]} 0 0
[junit4] 2> 2563506 INFO (qtp1207983899-13198) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[5 (1570029354111467520)]} 0 0
[junit4] 2> 2563507 INFO (qtp494089468-13184) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[6 (1570029354112516096)]} 0 0
[junit4] 2> 2563509 INFO (qtp1207983899-13201) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[6 (1570029354114613248)]} 0 0
[junit4] 2> 2563510 INFO (qtp494089468-13185) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[7 (1570029354115661824)]} 0 0
[junit4] 2> 2563511 INFO (qtp1207983899-13202) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[7 (1570029354115661824)]} 0 0
[junit4] 2> 2563512 INFO (qtp494089468-13186) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[8 (1570029354117758976)]} 0 0
[junit4] 2> 2563513 INFO (qtp1207983899-13203) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[8 (1570029354118807552)]} 0 0
[junit4] 2> 2563514 INFO (qtp494089468-13187) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[9 (1570029354119856128)]} 0 0
[junit4] 2> 2563515 INFO (qtp1207983899-13203) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[9 (1570029354119856128)]} 0 0
[junit4] 2> 2563515 INFO (qtp494089468-13188) [ x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1570029354120904704,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2563515 INFO (qtp494089468-13188) [ x:collection1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@6b662ce2 commitCommandVersion:1570029354120904704
[junit4] 2> 2563535 INFO (qtp494089468-13188) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5185a276[collection1] main]
[junit4] 2> 2563538 INFO (searcherExecutor-3751-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5185a276[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):c9)))}
[junit4] 2> 2563538 INFO (qtp494089468-13188) [ x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2563538 INFO (qtp494089468-13188) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 22
[junit4] 2> 2563539 INFO (qtp1207983899-13203) [ x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1570029354146070528,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2563539 INFO (qtp1207983899-13203) [ x:collection1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@3d24057f commitCommandVersion:1570029354146070528
[junit4] 2> 2563547 INFO (qtp1207983899-13203) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@682c3d6f[collection1] main]
[junit4] 2> 2563549 INFO (searcherExecutor-3762-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@682c3d6f[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):c9)))}
[junit4] 2> 2563549 INFO (qtp1207983899-13203) [ x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2563549 INFO (qtp1207983899-13203) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 10
[junit4] 2> 2563552 INFO (qtp494089468-13181) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[10 (1570029354157604864)]} 0 2
[junit4] 2> 2563555 INFO (qtp1207983899-13198) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[10 (1570029354160750592)]} 0 1
[junit4] 2> 2563556 INFO (qtp494089468-13184) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[11 (1570029354162847744)]} 0 0
[junit4] 2> 2563557 INFO (qtp1207983899-13201) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[11 (1570029354163896320)]} 0 0
[junit4] 2> 2563558 INFO (qtp494089468-13185) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[12 (1570029354164944896)]} 0 0
[junit4] 2> 2563558 INFO (qtp1207983899-13202) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[12 (1570029354165993472)]} 0 0
[junit4] 2> 2563559 INFO (qtp494089468-13186) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[13 (1570029354167042048)]} 0 0
[junit4] 2> 2563560 INFO (qtp1207983899-13205) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[13 (1570029354167042048)]} 0 0
[junit4] 2> 2563561 INFO (qtp494089468-13186) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[1001 (1570029354168090624)]} 0 0
[junit4] 2> 2563562 INFO (qtp1207983899-13204) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[1001 (1570029354169139200)]} 0 0
[junit4] 2> 2563563 INFO (qtp494089468-13188) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[14 (1570029354170187776)]} 0 0
[junit4] 2> 2563564 INFO (qtp1207983899-13203) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[14 (1570029354171236352)]} 0 0
[junit4] 2> 2563564 INFO (qtp494089468-13181) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[15 (1570029354172284928)]} 0 0
[junit4] 2> 2563565 INFO (qtp1207983899-13198) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[15 (1570029354173333504)]} 0 0
[junit4] 2> 2563566 INFO (qtp494089468-13181) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[16 (1570029354173333504)]} 0 0
[junit4] 2> 2563567 INFO (qtp1207983899-13201) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[16 (1570029354174382080)]} 0 0
[junit4] 2> 2563568 INFO (qtp494089468-13185) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[17 (1570029354175430656)]} 0 0
[junit4] 2> 2563569 INFO (qtp1207983899-13201) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[17 (1570029354176479232)]} 0 0
[junit4] 2> 2563570 INFO (qtp494089468-13187) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[100 (1570029354177527808)]} 0 0
[junit4] 2> 2563571 INFO (qtp1207983899-13205) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[100 (1570029354178576384)]} 0 0
[junit4] 2> 2563571 INFO (qtp494089468-13186) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[101 (1570029354179624960)]} 0 0
[junit4] 2> 2563572 INFO (qtp1207983899-13204) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[101 (1570029354180673536)]} 0 0
[junit4] 2> 2563573 INFO (qtp494089468-13188) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[102 (1570029354180673536)]} 0 0
[junit4] 2> 2563574 INFO (qtp1207983899-13203) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[102 (1570029354181722112)]} 0 0
[junit4] 2> 2563575 INFO (qtp494089468-13184) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[103 (1570029354182770688)]} 0 0
[junit4] 2> 2563575 INFO (qtp1207983899-13198) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[103 (1570029354183819264)]} 0 0
[junit4] 2> 2563576 INFO (qtp494089468-13184) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[104 (1570029354184867840)]} 0 0
[junit4] 2> 2563577 INFO (qtp1207983899-13202) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[104 (1570029354185916416)]} 0 0
[junit4] 2> 2563578 INFO (qtp494089468-13185) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[105 (1570029354186964992)]} 0 0
[junit4] 2> 2563579 INFO (qtp1207983899-13201) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[105 (1570029354186964992)]} 0 0
[junit4] 2> 2563579 INFO (qtp494089468-13185) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[106 (1570029354188013568)]} 0 0
[junit4] 2> 2563580 INFO (qtp1207983899-13205) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[106 (1570029354189062144)]} 0 0
[junit4] 2> 2563581 INFO (qtp494089468-13186) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[107 (1570029354190110720)]} 0 0
[junit4] 2> 2563582 INFO (qtp1207983899-13205) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[107 (1570029354190110720)]} 0 0
[junit4] 2> 2563582 INFO (qtp494089468-13186) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[108 (1570029354191159296)]} 0 0
[junit4] 2> 2563583 INFO (qtp1207983899-13203) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[108 (1570029354192207872)]} 0 0
[junit4] 2> 2563584 INFO (qtp494089468-13186) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[109 (1570029354193256448)]} 0 0
[junit4] 2> 2563585 INFO (qtp1207983899-13198) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[109 (1570029354193256448)]} 0 0
[junit4] 2> 2563586 INFO (qtp494089468-13184) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[110 (1570029354194305024)]} 0 0
[junit4] 2> 2563586 INFO (qtp1207983899-13202) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[110 (1570029354195353600)]} 0 0
[junit4] 2> 2563587 INFO (qtp494089468-13184) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[111 (1570029354196402176)]} 0 0
[junit4] 2> 2563588 INFO (qtp1207983899-13202) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[111 (1570029354196402176)]} 0 0
[junit4] 2> 2563588 INFO (qtp494089468-13185) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[112 (1570029354197450752)]} 0 0
[junit4] 2> 2563589 INFO (qtp1207983899-13204) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[112 (1570029354198499328)]} 0 0
[junit4] 2> 2563590 INFO (qtp494089468-13188) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[113 (1570029354198499328)]} 0 0
[junit4] 2> 2563590 INFO (qtp1207983899-13205) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[113 (1570029354199547904)]} 0 0
[junit4] 2> 2563591 INFO (qtp494089468-13188) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[114 (1570029354200596480)]} 0 0
[junit4] 2> 2563592 INFO (qtp1207983899-13203) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[114 (1570029354201645056)]} 0 0
[junit4] 2> 2563593 INFO (qtp494089468-13188) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[115 (1570029354201645056)]} 0 0
[junit4] 2> 2563593 INFO (qtp1207983899-13198) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[115 (1570029354202693632)]} 0 0
[junit4] 2> 2563594 INFO (qtp494089468-13187) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[116 (1570029354203742208)]} 0 0
[junit4] 2> 2563595 INFO (qtp1207983899-13201) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[116 (1570029354203742208)]} 0 0
[junit4] 2> 2563595 INFO (qtp494089468-13184) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[117 (1570029354204790784)]} 0 0
[junit4] 2> 2563596 INFO (qtp1207983899-13201) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[117 (1570029354205839360)]} 0 0
[junit4] 2> 2563597 INFO (qtp494089468-13185) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[118 (1570029354205839360)]} 0 0
[junit4] 2> 2563597 INFO (qtp1207983899-13204) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[118 (1570029354206887936)]} 0 0
[junit4] 2> 2563598 INFO (qtp494089468-13181) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[119 (1570029354207936512)]} 0 0
[junit4] 2> 2563599 INFO (qtp1207983899-13205) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[119 (1570029354207936512)]} 0 0
[junit4] 2> 2563600 INFO (qtp494089468-13181) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[120 (1570029354208985088)]} 0 0
[junit4] 2> 2563600 INFO (qtp1207983899-13203) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[120 (1570029354210033664)]} 0 0
[junit4] 2> 2563601 INFO (qtp494089468-13181) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[121 (1570029354211082240)]} 0 0
[junit4] 2> 2563602 INFO (qtp1207983899-13198) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[121 (1570029354211082240)]} 0 0
[junit4] 2> 2563603 INFO (qtp494089468-13187) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[122 (1570029354212130816)]} 0 0
[junit4] 2> 2563603 INFO (qtp1207983899-13198) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[122 (1570029354213179392)]} 0 0
[junit4] 2> 2563604 INFO (qtp494089468-13184) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[123 (1570029354214227968)]} 0 0
[junit4] 2> 2563605 INFO (qtp1207983899-13198) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[123 (1570029354214227968)]} 0 0
[junit4] 2> 2563605 INFO (qtp494089468-13185) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[124 (1570029354215276544)]} 0 0
[junit4] 2> 2563606 INFO (qtp1207983899-13198) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[124 (1570029354216325120)]} 0 0
[junit4] 2> 2563607 INFO (qtp494089468-13186) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[125 (1570029354216325120)]} 0 0
[junit4] 2> 2563607 INFO (qtp1207983899-13205) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[125 (1570029354217373696)]} 0 0
[junit4] 2> 2563608 INFO (qtp494089468-13188) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[126 (1570029354218422272)]} 0 0
[junit4] 2> 2563609 INFO (qtp1207983899-13203) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[126 (1570029354218422272)]} 0 0
[junit4] 2> 2563610 INFO (qtp494089468-13188) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[127 (1570029354219470848)]} 0 0
[junit4] 2> 2563611 INFO (qtp1207983899-13202) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[127 (1570029354220519424)]} 0 0
[junit4] 2> 2563611 INFO (qtp494089468-13187) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[128 (1570029354221568000)]} 0 0
[junit4] 2> 2563612 INFO (qtp1207983899-13201) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[128 (1570029354222616576)]} 0 0
[junit4] 2> 2563613 INFO (qtp494089468-13187) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[129 (1570029354223665152)]} 0 0
[junit4] 2> 2563614 INFO (qtp1207983899-13201) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[129 (1570029354223665152)]} 0 0
[junit4] 2> 2563614 INFO (qtp494089468-13185) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[130 (1570029354224713728)]} 0 0
[junit4] 2> 2563615 INFO (qtp1207983899-13198) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[130 (1570029354225762304)]} 0 0
[junit4] 2> 2563616 INFO (qtp494089468-13186) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[131 (1570029354225762304)]} 0 0
[junit4] 2> 2563616 INFO (qtp1207983899-13198) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[131 (1570029354226810880)]} 0 0
[junit4] 2> 2563617 INFO (qtp494089468-13181) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[132 (1570029354227859456)]} 0 0
[junit4] 2> 2563624 INFO (qtp1207983899-13203) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[132 (1570029354235199488)]} 0 0
[junit4] 2> 2563625 INFO (qtp494089468-13188) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[133 (1570029354236248064)]} 0 0
[junit4] 2> 2563626 INFO (qtp1207983899-13202) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[133 (1570029354237296640)]} 0 0
[junit4] 2> 2563627 INFO (qtp494089468-13188) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[134 (1570029354237296640)]} 0 0
[junit4] 2> 2563627 INFO (qtp1207983899-13202) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[134 (1570029354238345216)]} 0 0
[junit4] 2> 2563628 INFO (qtp494089468-13188) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[135 (1570029354239393792)]} 0 0
[junit4] 2> 2563629 INFO (qtp1207983899-13201) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[135 (1570029354239393792)]} 0 0
[junit4] 2> 2563629 INFO (qtp494089468-13185) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[136 (1570029354240442368)]} 0 0
[junit4] 2> 2563630 INFO (qtp1207983899-13201) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[136 (1570029354241490944)]} 0 0
[junit4] 2> 2563631 INFO (qtp494089468-13186) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[137 (1570029354241490944)]} 0 0
[junit4] 2> 2563631 INFO (qtp1207983899-13201) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[137 (1570029354242539520)]} 0 0
[junit4] 2> 2563632 INFO (qtp494089468-13181) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[138 (1570029354243588096)]} 0 0
[junit4] 2> 2563633 INFO (qtp1207983899-13203) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[138 (1570029354243588096)]} 0 0
[junit4] 2> 2563633 INFO (qtp494089468-13184) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[139 (1570029354244636672)]} 0 0
[junit4] 2> 2563634 INFO (qtp1207983899-13204) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[139 (1570029354245685248)]} 0 0
[junit4] 2> 2563635 INFO (qtp494089468-13184) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[140 (1570029354245685248)]} 0 0
[junit4] 2> 2563635 INFO (qtp1207983899-13204) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[140 (1570029354246733824)]} 0 0
[junit4] 2> 2563636 INFO (qtp494089468-13184) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[141 (1570029354247782400)]} 0 0
[junit4] 2> 2563637 INFO (qtp1207983899-13205) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[141 (1570029354247782400)]} 0 0
[junit4] 2> 2563637 INFO (qtp494089468-13185) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[142 (1570029354248830976)]} 0 0
[junit4] 2> 2563638 INFO (qtp1207983899-13198) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[142 (1570029354249879552)]} 0 0
[junit4] 2> 2563639 INFO (qtp494089468-13185) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[143 (1570029354249879552)]} 0 0
[junit4] 2> 2563639 INFO (qtp1207983899-13201) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[143 (1570029354250928128)]} 0 0
[junit4] 2> 2563640 INFO (qtp494089468-13181) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[144 (1570029354251976704)]} 0 0
[junit4] 2> 2563641 INFO (qtp1207983899-13201) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[144 (1570029354251976704)]} 0 0
[junit4] 2> 2563641 INFO (qtp494089468-13181) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[145 (1570029354253025280)]} 0 0
[junit4] 2> 2563642 INFO (qtp1207983899-13202) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[145 (1570029354254073856)]} 0 0
[junit4] 2> 2563643 INFO (qtp494089468-13181) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[146 (1570029354254073856)]} 0 0
[junit4] 2> 2563643 INFO (qtp1207983899-13202) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[146 (1570029354255122432)]} 0 0
[junit4] 2> 2563644 INFO (qtp494089468-13181) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[147 (1570029354256171008)]} 0 0
[junit4] 2> 2563645 INFO (qtp1207983899-13205) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[147 (1570029354256171008)]} 0 0
[junit4] 2> 2563645 INFO (qtp494089468-13186) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[148 (1570029354257219584)]} 0 0
[junit4] 2> 2563646 INFO (qtp1207983899-13198) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[148 (1570029354258268160)]} 0 0
[junit4] 2> 2563647 INFO (qtp494089468-13185) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[149 (1570029354258268160)]} 0 0
[junit4] 2> 2563647 INFO (qtp1207983899-13198) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={wt=javabin&version=2}{add=[149 (1570029354259316736)]} 0 0
[junit4] 2> 2563648 INFO (qtp494089468-13185) [ x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1570029354260365312,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2563648 INFO (qtp494089468-13185) [ x:collection1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@6b662ce2 commitCommandVersion:1570029354260365312
[junit4] 2> 2563661 INFO (qtp494089468-13185) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@436f7792[collection1] main]
[junit4] 2> 2563662 INFO (qtp494089468-13185) [ x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2563663 INFO (searcherExecutor-3751-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@436f7792[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):c9) Uninverting(_1(7.0.0):c59)))}
[junit4] 2> 2563663 INFO (qtp494089468-13185) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 15
[junit4] 2> 2563664 INFO (qtp1207983899-13198) [ x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1570029354277142528,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2563664 INFO (qtp1207983899-13198) [ x:collection1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@3d24057f commitCommandVersion:1570029354277142528
[junit4] 2> 2563676 INFO (qtp1207983899-13198) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@25ecef88[collection1] main]
[junit4] 2> 2563678 INFO (searcherExecutor-3762-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@25ecef88[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):c9) Uninverting(_1(7.0.0):c59)))}
[junit4] 2> 2563679 INFO (qtp1207983899-13198) [ x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2563679 INFO (qtp1207983899-13198) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp= path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 14
[junit4] 2> 2563680 INFO (qtp494089468-13188) [ x:collection1] o.a.s.h.RequestHandlerBase Ignoring exception matching parameter cannot be negative
[junit4] 2> 2563680 INFO (qtp494089468-13188) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*&distrib=false&start=-1&wt=javabin&version=2} status=400 QTime=0
[junit4] 2> 2563681 INFO (qtp494089468-13188) [ x:collection1] o.a.s.h.RequestHandlerBase Ignoring exception matching parameter cannot be negative
[junit4] 2> 2563681 INFO (qtp494089468-13188) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*&distrib=false&start=0&rows=-1&wt=javabin&version=2} status=400 QTime=0
[junit4] 2> 2563682 INFO (qtp494089468-13181) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&distrib=false&sort=n_ti1+desc&wt=javabin&version=2} hits=68 status=0 QTime=0
[junit4] 2> 2563738 INFO (qtp1207983899-13199) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={df=text&distrib=false&fl=id&shards.purpose=4&start=0&fsv=true&sort=n_ti1+desc&shard.url=[ff01::083]:33332/|http://127.0.0.1:60942//collection1|[ff01::114]:33332/|[ff01::213]:33332/&rows=10&version=2&q=*:*&NOW=1497296671198&isShard=true&wt=javabin} hits=68 status=0 QTime=0
[junit4] 2> 2563743 INFO (qtp1207983899-13205) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&df=text&distrib=false&shards.purpose=64&NOW=1497296671198&ids=145,14,113,136,125,126,115,105,138,8&isShard=true&shard.url=[ff01::083]:33332/|http://127.0.0.1:60942//collection1|[ff01::114]:33332/|[ff01::213]:33332/&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2563749 INFO (qtp1207983899-13204) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&shards=[ff01::083]:33332/|http://127.0.0.1:60942//collection1|[ff01::114]:33332/|[ff01::213]:33332/&sort=n_ti1+desc&wt=javabin&version=2} hits=68 status=0 QTime=64
[junit4] 2> 2563749 INFO (TEST-TestDistributedSearch.test-seed#[96F460804B43D487]) [ ] o.a.s.BaseDistributedSearchTestCase starting stress...
[junit4] 2> 2563753 INFO (qtp1207983899-13199) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={df=text&distrib=false&fl=id&shards.purpose=4&start=0&fsv=true&sort=n_ti1+desc&shard.url=[ff01::083]:33332/|http://127.0.0.1:60942//collection1|[ff01::114]:33332/|[ff01::213]:33332/&rows=10&version=2&q=*:*&NOW=1497296671266&isShard=true&wt=javabin} hits=68 status=0 QTime=0
[junit4] 2> 2563754 INFO (qtp1207983899-13201) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={df=text&distrib=false&fl=id&shards.purpose=4&start=0&fsv=true&sort=n_ti1+desc&shard.url=[ff01::083]:33332/|http://127.0.0.1:60942//collection1|[ff01::114]:33332/|[ff01::213]:33332/&rows=10&version=2&q=*:*&NOW=1497296671267&isShard=true&wt=javabin} hits=68 status=0 QTime=0
[junit4] 2> 2563754 INFO (qtp1207983899-13198) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={df=text&distrib=false&fl=id&shards.purpose=4&start=0&fsv=true&sort=n_ti1+desc&shard.url=[ff01::083]:33332/|http://127.0.0.1:60942//collection1|[ff01::114]:33332/|[ff01::213]:33332/&rows=10&version=2&q=*:*&NOW=1497296671268&isShard=true&wt=javabin} hits=68 status=0 QTime=0
[junit4] 2> 2563755 INFO (qtp1207983899-13205) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&df=text&distrib=false&shards.purpose=64&NOW=1497296671268&ids=145,14,113,136,125,126,115,105,138,8&isShard=true&shard.url=[ff01::083]:33332/|http://127.0.0.1:60942//collection1|[ff01::114]:33332/|[ff01::213]:33332/&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2563756 INFO (qtp1207983899-13201) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&df=text&distrib=false&shards.purpose=64&NOW=1497296671267&ids=145,14,113,136,125,126,115,105,138,8&isShard=true&shard.url=[ff01::083]:33332/|http://127.0.0.1:60942//collection1|[ff01::114]:33332/|[ff01::213]:33332/&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2563757 INFO (qtp1207983899-13203) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&shards=[ff01::083]:33332/|http://127.0.0.1:60942//collection1|[ff01::114]:33332/|[ff01::213]:33332/&sort=n_ti1+desc&wt=javabin&version=2} hits=68 status=0 QTime=4
[junit4] 2> 2563758 INFO (qtp1207983899-13204) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&shards=[ff01::083]:33332/|http://127.0.0.1:60942//collection1|[ff01::114]:33332/|[ff01::213]:33332/&sort=n_ti1+desc&wt=javabin&version=2} hits=68 status=0 QTime=3
[junit4] 2> 2563759 INFO (qtp1207983899-13205) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&df=text&distrib=false&shards.purpose=64&NOW=1497296671266&ids=145,14,113,136,125,126,115,105,138,8&isShard=true&shard.url=[ff01::083]:33332/|http://127.0.0.1:60942//collection1|[ff01::114]:33332/|[ff01::213]:33332/&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2563760 INFO (qtp1207983899-13201) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={df=text&distrib=false&fl=id&shards.purpose=4&start=0&fsv=true&sort=n_ti1+desc&shard.url=[ff01::083]:33332/|http://127.0.0.1:60942//collection1|[ff01::114]:33332/|[ff01::213]:33332/&rows=10&version=2&q=*:*&NOW=1497296671272&isShard=true&wt=javabin} hits=68 status=0 QTime=0
[junit4] 2> 2563761 INFO (qtp1207983899-13202) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&shards=[ff01::083]:33332/|http://127.0.0.1:60942//collection1|[ff01::114]:33332/|[ff01::213]:33332/&sort=n_ti1+desc&wt=javabin&version=2} hits=68 status=0 QTime=8
[junit4] 2> 2563763 INFO (qtp1207983899-13204) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&df=text&distrib=false&shards.purpose=64&NOW=1497296671272&ids=145,14,113,136,125,126,115,105,138,8&isShard=true&shard.url=[ff01::083]:33332/|http://127.0.0.1:60942//collection1|[ff01::114]:33332/|[ff01::213]:33332/&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2563763 INFO (qtp1207983899-13204) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={df=text&distrib=false&fl=id&shards.purpose=4&start=0&fsv=true&sort=n_ti1+desc&shard.url=[ff01::083]:33332/|http://127.0.0.1:60942//collection1|[ff01::114]:33332/|[ff01::213]:33332/&rows=10&version=2&q=*:*&NOW=1497296671273&isShard=true&wt=javabin} hits=68 status=0 QTime=0
[junit4] 2> 2563764 INFO (qtp1207983899-13198) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={df=text&distrib=false&fl=id&shards.purpose=4&start=0&fsv=true&sort=n_ti1+desc&shard.url=[ff01::083]:33332/|http://127.0.0.1:60942//collection1|[ff01::114]:33332/|[ff01::213]:33332/&rows=10&version=2&q=*:*&NOW=1497296671276&isShard=true&wt=javabin} hits=68 status=0 QTime=0
[junit4] 2> 2563764 INFO (qtp1207983899-13199) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&shards=[ff01::083]:33332/|http://127.0.0.1:60942//collection1|[ff01::114]:33332/|[ff01::213]:33332/&sort=n_ti1+desc&wt=javabin&version=2} hits=68 status=0 QTime=6
[junit4] 2> 2563765 INFO (qtp1207983899-13204) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&df=text&distrib=false&shards.purpose=64&NOW=1497296671273&ids=145,14,113,136,125,126,115,105,138,8&isShard=true&shard.url=[ff01::083]:33332/|http://127.0.0.1:60942//collection1|[ff01::114]:33332/|[ff01::213]:33332/&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2563766 INFO (qtp1207983899-13205) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&shards=[ff01::083]:33332/|http://127.0.0.1:60942//collection1|[ff01::114]:33332/|[ff01::213]:33332/&sort=n_ti1+desc&wt=javabin&version=2} hits=68 status=0 QTime=6
[junit4] 2> 2563767 INFO (qtp1207983899-13198) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&df=text&distrib=false&shards.purpose=64&NOW=1497296671276&ids=145,14,113,136,125,126,115,105,138,8&isShard=true&shard.url=[ff01::083]:33332/|http://127.0.0.1:60942//collection1|[ff01::114]:33332/|[ff01::213]:33332/&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2563768 INFO (qtp1207983899-13201) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&shards=[ff01::083]:33332/|http://127.0.0.1:60942//collection1|[ff01::114]:33332/|[ff01::213]:33332/&sort=n_ti1+desc&wt=javabin&version=2} hits=68 status=0 QTime=6
[junit4] 2> 2563768 INFO (qtp494089468-13186) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&distrib=false&sort=n_ti1+asc&wt=javabin&version=2} hits=68 status=0 QTime=0
[junit4] 2> 2563796 INFO (qtp1207983899-13224) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={df=text&distrib=false&fl=id&shards.purpose=4&start=0&fsv=true&sort=n_ti1+asc&shard.url=[ff01::114]:33332/|http://127.0.0.1:60942//collection1|[ff01::213]:33332/&rows=10&version=2&q=*:*&NOW=1497296671284&isShard=true&wt=javabin} hits=68 status=0 QTime=0
[junit4] 2> 2563798 INFO (qtp1207983899-13199) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&df=text&distrib=false&shards.purpose=64&NOW=1497296671284&ids=143,121,122,2,3,103,127,116,117,131&isShard=true&shard.url=[ff01::114]:33332/|http://127.0.0.1:60942//collection1|[ff01::213]:33332/&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2563806 INFO (qtp1207983899-13198) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&shards=[ff01::114]:33332/|http://127.0.0.1:60942//collection1|[ff01::213]:33332/&sort=n_ti1+asc&wt=javabin&version=2} hits=68 status=0 QTime=36
[junit4] 2> 2563807 INFO (TEST-TestDistributedSearch.test-seed#[96F460804B43D487]) [ ] o.a.s.BaseDistributedSearchTestCase starting stress...
[junit4] 2> 2563808 INFO (qtp1207983899-13202) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={df=text&distrib=false&fl=id&shards.purpose=4&start=0&fsv=true&sort=n_ti1+asc&shard.url=[ff01::114]:33332/|http://127.0.0.1:60942//collection1|[ff01::213]:33332/&rows=10&version=2&q=*:*&NOW=1497296671321&isShard=true&wt=javabin} hits=68 status=0 QTime=0
[junit4] 2> 2563808 INFO (qtp1207983899-13205) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={df=text&distrib=false&fl=id&shards.purpose=4&start=0&fsv=true&sort=n_ti1+asc&shard.url=[ff01::114]:33332/|http://127.0.0.1:60942//collection1|[ff01::213]:33332/&rows=10&version=2&q=*:*&NOW=1497296671321&isShard=true&wt=javabin} hits=68 status=0 QTime=0
[junit4] 2> 2563809 INFO (qtp1207983899-13205) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&df=text&distrib=false&shards.purpose=64&NOW=1497296671321&ids=143,121,122,2,3,103,127,116,117,131&isShard=true&shard.url=[ff01::114]:33332/|http://127.0.0.1:60942//collection1|[ff01::213]:33332/&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2563809 INFO (qtp1207983899-13225) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={df=text&distrib=false&fl=id&shards.purpose=4&start=0&fsv=true&sort=n_ti1+asc&shard.url=[ff01::114]:33332/|http://127.0.0.1:60942//collection1|[ff01::213]:33332/&rows=10&version=2&q=*:*&NOW=1497296671323&isShard=true&wt=javabin} hits=68 status=0 QTime=0
[junit4] 2> 2563810 INFO (qtp1207983899-13225) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&df=text&distrib=false&shards.purpose=64&NOW=1497296671321&ids=143,121,122,2,3,103,127,116,117,131&isShard=true&shard.url=[ff01::114]:33332/|http://127.0.0.1:60942//collection1|[ff01::213]:33332/&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2563811 INFO (qtp1207983899-13227) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&shards=[ff01::114]:33332/|http://127.0.0.1:60942//collection1|[ff01::213]:33332/&sort=n_ti1+asc&wt=javabin&version=2} hits=68 status=0 QTime=3
[junit4] 2> 2563811 INFO (qtp1207983899-13204) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&shards=[ff01::114]:33332/|http://127.0.0.1:60942//collection1|[ff01::213]:33332/&sort=n_ti1+asc&wt=javabin&version=2} hits=68 status=0 QTime=3
[junit4] 2> 2563812 INFO (qtp1207983899-13224) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&df=text&distrib=false&shards.purpose=64&NOW=1497296671323&ids=143,121,122,2,3,103,127,116,117,131&isShard=true&shard.url=[ff01::114]:33332/|http://127.0.0.1:60942//collection1|[ff01::213]:33332/&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2563813 INFO (qtp1207983899-13202) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&shards=[ff01::114]:33332/|http://127.0.0.1:60942//collection1|[ff01::213]:33332/&sort=n_ti1+asc&wt=javabin&version=2} hits=68 status=0 QTime=4
[junit4] 2> 2563816 INFO (qtp1207983899-13201) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={df=text&distrib=false&fl=id&shards.purpose=4&start=0&fsv=true&sort=n_ti1+asc&shard.url=[ff01::114]:33332/|http://127.0.0.1:60942//collection1|[ff01::213]:33332/&rows=10&version=2&q=*:*&NOW=1497296671327&isShard=true&wt=javabin} hits=68 status=0 QTime=0
[junit4] 2> 2563817 INFO (qtp1207983899-13225) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={df=text&distrib=false&fl=id&shards.purpose=4&start=0&fsv=true&sort=n_ti1+asc&shard.url=[ff01::114]:33332/|http://127.0.0.1:60942//collection1|[ff01::213]:33332/&rows=10&version=2&q=*:*&NOW=1497296671330&isShard=true&wt=javabin} hits=68 status=0 QTime=0
[junit4] 2> 2563818 INFO (qtp1207983899-13201) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={df=text&distrib=false&fl=id&shards.purpose=4&start=0&fsv=true&sort=n_ti1+asc&shard.url=[ff01::114]:33332/|http://127.0.0.1:60942//collection1|[ff01::213]:33332/&rows=10&version=2&q=*:*&NOW=1497296671331&isShard=true&wt=javabin} hits=68 status=0 QTime=0
[junit4] 2> 2563818 INFO (qtp1207983899-13225) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&df=text&distrib=false&shards.purpose=64&NOW=1497296671327&ids=143,121,122,2,3,103,127,116,117,131&isShard=true&shard.url=[ff01::114]:33332/|http://127.0.0.1:60942//collection1|[ff01::213]:33332/&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2563818 INFO (qtp1207983899-13199) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&shards=[ff01::114]:33332/|http://127.0.0.1:60942//collection1|[ff01::213]:33332/&sort=n_ti1+asc&wt=javabin&version=2} hits=68 status=0 QTime=5
[junit4] 2> 2563819 INFO (qtp1207983899-13225) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&df=text&distrib=false&shards.purpose=64&NOW=1497296671331&ids=143,121,122,2,3,103,127,116,117,131&isShard=true&shard.url=[ff01::114]:33332/|http://127.0.0.1:60942//collection1|[ff01::213]:33332/&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2563820 INFO (qtp1207983899-13199) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&df=text&distrib=false&shards.purpose=64&NOW=1497296671330&ids=143,121,122,2,3,103,127,116,117,131&isShard=true&shard.url=[ff01::114]:33332/|http://127.0.0.1:60942//collection1|[ff01::213]:33332/&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2563820 INFO (qtp1207983899-13204) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&shards=[ff01::114]:33332/|http://127.0.0.1:60942//collection1|[ff01::213]:33332/&sort=n_ti1+asc&wt=javabin&version=2} hits=68 status=0 QTime=3
[junit4] 2> 2563820 INFO (qtp1207983899-13205) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&shards=[ff01::114]:33332/|http://127.0.0.1:60942//collection1|[ff01::213]:33332/&sort=n_ti1+asc&wt=javabin&version=2} hits=68 status=0 QTime=4
[junit4] 2> 2563822 INFO (qtp494089468-13187) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&distrib=false&sort=n_f1+desc&wt=javabin&version=2} hits=68 status=0 QTime=0
[junit4] 2> 2563824 INFO (qtp1207983899-13202) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={df=text&distrib=false&fl=id&shards.purpose=4&start=0&fsv=true&sort=n_f1+desc&shard.url=[ff01::114]:33332/|http://127.0.0.1:60942//collection1|[ff01::213]:33332/&rows=10&version=2&q=*:*&NOW=1497296671337&isShard=true&wt=javabin} hits=68 status=0 QTime=0
[junit4] 2> 2563825 INFO (qtp1207983899-13202) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&df=text&distrib=false&shards.purpose=64&NOW=1497296671337&ids=132,143,133,134,15,4,5,138,107,131&isShard=true&shard.url=[ff01::114]:33332/|http://127.0.0.1:60942//collection1|[ff01::213]:33332/&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2563826 INFO (qtp1207983899-13201) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&shards=[ff01::114]:33332/|http://127.0.0.1:60942//collection1|[ff01::213]:33332/&sort=n_f1+desc&wt=javabin&version=2} hits=68 status=0 QTime=3
[junit4] 2> 2563827 INFO (TEST-TestDistributedSearch.test-seed#[96F460804B43D487]) [ ] o.a.s.BaseDistributedSearchTestCase starting stress...
[junit4] 2> 2563828 INFO (qtp1207983899-13202) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={df=text&distrib=false&fl=id&shards.purpose=4&start=0&fsv=true&sort=n_f1+desc&shard.url=[ff01::114]:33332/|http://127.0.0.1:60942//collection1|[ff01::213]:33332/&rows=10&version=2&q=*:*&NOW=1497296671341&isShard=true&wt=javabin} hits=68 status=0 QTime=0
[junit4] 2> 2563829 INFO (qtp1207983899-13202) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&df=text&distrib=false&shards.purpose=64&NOW=1497296671341&ids=132,143,133,134,15,4,5,138,107,131&isShard=true&shard.url=[ff01::114]:33332/|http://127.0.0.1:60942//collection1|[ff01::213]:33332/&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2563829 INFO (qtp1207983899-13227) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&shards=[ff01::114]:33332/|http://127.0.0.1:60942//collection1|[ff01::213]:33332/&sort=n_f1+desc&wt=javabin&version=2} hits=68 status=0 QTime=2
[junit4] 2> 2563830 INFO (qtp1207983899-13198) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={df=text&distrib=false&fl=id&shards.purpose=4&start=0&fsv=true&sort=n_f1+desc&shard.url=[ff01::114]:33332/|http://127.0.0.1:60942//collection1|[ff01::213]:33332/&rows=10&version=2&q=*:*&NOW=1497296671344&isShard=true&wt=javabin} hits=68 status=0 QTime=0
[junit4] 2> 2563831 INFO (qtp1207983899-13198) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={df=text&distrib=false&fl=id&shards.purpose=4&start=0&fsv=true&sort=n_f1+desc&shard.url=[ff01::114]:33332/|http://127.0.0.1:60942//collection1|[ff01::213]:33332/&rows=10&version=2&q=*:*&NOW=1497296671344&isShard=true&wt=javabin} hits=68 status=0 QTime=0
[junit4] 2> 2563831 INFO (qtp1207983899-13198) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&df=text&distrib=false&shards.purpose=64&NOW=1497296671344&ids=132,143,133,134,15,4,5,138,107,131&isShard=true&shard.url=[ff01::114]:33332/|http://127.0.0.1:60942//collection1|[ff01::213]:33332/&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2563832 INFO (qtp1207983899-13198) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&df=text&distrib=false&shards.purpose=64&NOW=1497296671344&ids=132,143,133,134,15,4,5,138,107,131&isShard=true&shard.url=[ff01::114]:33332/|http://127.0.0.1:60942//collection1|[ff01::213]:33332/&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2563833 INFO (qtp1207983899-13225) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&shards=[ff01::114]:33332/|http://127.0.0.1:60942//collection1|[ff01::213]:33332/&sort=n_f1+desc&wt=javabin&version=2} hits=68 status=0 QTime=2
[junit4] 2> 2563833 INFO (qtp1207983899-13204) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={q=*:*&shards=[ff01::114]:33332/|http://127.0.0.1:60942//collection1|[ff01::213]:33332/&sort=n_f1+desc&wt=javabin&version=2} hits=68 status=0 QTime=3
[junit4] 2> 2563834 INFO (qtp1207983899-13204) [ x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/select params={df=text&distrib=false&fl=id&shards.purpose=4&start=0&fsv=true&sort=n_f1+desc&shard.url=[ff01::114]:33332/|http://127.0.0.1:60942//collection1|[ff01::213]:33332/&rows=10&version=2&q=*:*&NOW=1497296671347&isShard=true&wt=javab
[...truncated too long message...]
ection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=1315432679
[junit4] 2> 2701821 INFO (TEST-TestDistributedSearch.test-seed#[96F460804B43D487]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@24b02c83{/,null,UNAVAILABLE}
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestDistributedSearch -Dtests.method=test -Dtests.seed=96F460804B43D487 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=en-ZA -Dtests.timezone=America/Merida -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] ERROR 140s J0 | TestDistributedSearch.test <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=44906, name=Thread-33143, state=RUNNABLE, group=TGRP-TestDistributedSearch]
[junit4] > at __randomizedtesting.SeedInfo.seed([96F460804B43D487:1EA05F5AE5BFB97F]:0)
[junit4] > Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:39331//collection1: java.lang.NullPointerException
[junit4] > at org.apache.solr.search.grouping.distributed.responseprocessor.StoredFieldsShardResponseProcessor.process(StoredFieldsShardResponseProcessor.java:41)
[junit4] > at org.apache.solr.handler.component.QueryComponent.handleGroupedResponses(QueryComponent.java:768)
[junit4] > at org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:751)
[junit4] > at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:423)
[junit4] > at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:178)
[junit4] > at org.apache.solr.core.SolrCore.execute(SolrCore.java:2487)
[junit4] > at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:722)
[junit4] > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:528)
[junit4] > at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:374)
[junit4] > at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:318)
[junit4] > at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
[junit4] > at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] > at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
[junit4] > at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] > at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] > at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] > at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] > at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] > at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] > at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] > at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:462)
[junit4] > at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] > at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] > at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] > at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] > at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
[junit4] > at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
[junit4] > at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([96F460804B43D487]:0)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:610)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:237)
[junit4] > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
[junit4] > at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
[junit4] > at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:957)
[junit4] > at org.apache.solr.TestDistributedSearch$1.run(TestDistributedSearch.java:1140)
[junit4] 2> 2701826 INFO (SUITE-TestDistributedSearch-seed#[96F460804B43D487]-worker) [ ] o.a.s.SolrTestCaseJ4 ------------------------------------------------------- Done waiting for tracked resources to be released
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {other_tl1=PostingsFormat(name=Memory), range_facet_l_dv=FSTOrd50, SubjectTerms_mfacet=PostingsFormat(name=Memory), n_l1=PostingsFormat(name=Memory), a_n_tdt=FSTOrd50, multiDefault=BlockTreeOrds(blocksize=128), intDefault=PostingsFormat(name=Memory), n_td1=BlockTreeOrds(blocksize=128), n_d1=PostingsFormat(name=Memory), n_f1=BlockTreeOrds(blocksize=128), range_facet_l=PostingsFormat(name=Memory), n_tl1=BlockTreeOrds(blocksize=128), a_s=PostingsFormat(name=LuceneFixedGap), oddField_s=PostingsFormat(name=LuceneFixedGap), n_tf1=PostingsFormat(name=Memory), lowerfilt1and2=BlockTreeOrds(blocksize=128), a_t=PostingsFormat(name=Memory), lowerfilt=BlockTreeOrds(blocksize=128), id=FSTOrd50, text=PostingsFormat(name=LuceneFixedGap), timestamp=PostingsFormat(name=Memory), severity=PostingsFormat(name=LuceneFixedGap), foo_sev_enum=FSTOrd50, n_dt1=BlockTreeOrds(blocksize=128), b_n_tdt=PostingsFormat(name=Memory), n_ti1=FSTOrd50, rnd_b=FSTOrd50, foo_b=PostingsFormat(name=LuceneFixedGap), n_tdt1=PostingsFormat(name=Memory), foo_d=FSTOrd50, range_facet_i_dv=PostingsFormat(name=Memory), foo_f=PostingsFormat(name=LuceneFixedGap)}, docValues:{range_facet_l_dv=DocValuesFormat(name=Memory), foo_sev_enum=DocValuesFormat(name=Memory), _version_=DocValuesFormat(name=Asserting), range_facet_i_dv=DocValuesFormat(name=Asserting), intDvoDefault=DocValuesFormat(name=Direct), timestamp=DocValuesFormat(name=Asserting), a_i_p=DocValuesFormat(name=Lucene70)}, maxPointsInLeafNode=1808, maxMBSortInHeap=5.238133884416585, sim=RandomSimilarity(queryNorm=false): {}, locale=en-ZA, timezone=America/Merida
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_131 (64-bit)/cpus=4,threads=2,free=154826872,total=534773760
[junit4] 2> NOTE: All tests run in this JVM: [TestCrossCoreJoin, RAMDirectoryFactoryTest, QueryElevationComponentTest, ConnectionReuseTest, TestHttpShardHandlerFactory, StandardRequestHandlerTest, TimeZoneUtilsTest, TestFunctionQuery, CopyFieldTest, QueryEqualityTest, LeaderElectionIntegrationTest, HLLUtilTest, TestBackupRepositoryFactory, CurrencyFieldOpenExchangeTest, TestMaxScoreQueryParser, ParsingFieldUpdateProcessorsTest, DistributedVersionInfoTest, DocumentAnalysisRequestHandlerTest, TestCodecSupport, TestSmileRequest, TestReplicationHandler, TestFaceting, DistributedQueryComponentCustomSortTest, TestChildDocTransformer, SolrMetricsIntegrationTest, TestFoldingMultitermQuery, SecurityConfHandlerTest, CdcrRequestHandlerTest, TestCopyFieldCollectionResource, AnalysisAfterCoreReloadTest, TestIntervalFaceting, TestFieldCacheSort, BlobRepositoryCloudTest, ReturnFieldsTest, ShufflingReplicaListTransformerTest, TestFieldCacheWithThreads, DistributedFacetExistsSmallTest, DistributedQueryElevationComponentTest, UUIDFieldTest, TestAnalyzedSuggestions, JavabinLoaderTest, TestCoreAdminApis, DisMaxRequestHandlerTest, TestCollectionAPI, TestFieldSortValues, SpatialRPTFieldTypeTest, DistribDocExpirationUpdateProcessorTest, LeaderFailureAfterFreshStartTest, BasicDistributedZkTest, TestSQLHandlerNonCloud, PreAnalyzedFieldTest, InfoHandlerTest, TestHighFrequencyDictionaryFactory, CollectionTooManyReplicasTest, CloudMLTQParserTest, TestRealTimeGet, TestJsonFacetRefinement, TestApiFramework, ConfigureRecoveryStrategyTest, TestCSVLoader, TestDistributedSearch]
[junit4] Completed [442/725 (1!)] on J0 in 139.93s, 1 test, 1 error <<< FAILURES!
[...truncated 8420 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/build.xml:783: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/build.xml:727: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid30482.hprof
Total time: 292 minutes 22 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1322 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1322/
1 tests failed.
FAILED: org.apache.solr.core.TestJmxIntegration.testJmxRegistration
Error Message:
org.apache.lucene.store.AlreadyClosedException: this Directory is closed
Stack Trace:
javax.management.RuntimeMBeanException: org.apache.lucene.store.AlreadyClosedException: this Directory is closed
at __randomizedtesting.SeedInfo.seed([F5EFC91ED1D0469:818F98AB805C5C0C]:0)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrow(DefaultMBeanServerInterceptor.java:839)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrowMaybeMBeanException(DefaultMBeanServerInterceptor.java:852)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:651)
at com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678)
at org.apache.solr.core.TestJmxIntegration.testJmxRegistration(TestJmxIntegration.java:121)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.lucene.store.AlreadyClosedException: this Directory is closed
at org.apache.lucene.store.BaseDirectory.ensureOpen(BaseDirectory.java:51)
at org.apache.lucene.store.FSDirectory.listAll(FSDirectory.java:233)
at org.apache.solr.core.DirectoryFactory.sizeOfDirectory(DirectoryFactory.java:264)
at org.apache.solr.core.SolrCore.getIndexSize(SolrCore.java:419)
at org.apache.solr.handler.ReplicationHandler.lambda$initializeMetrics$2(ReplicationHandler.java:867)
at com.codahale.metrics.JmxReporter$JmxGauge.getValue(JmxReporter.java:188)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:71)
at sun.reflect.GeneratedMethodAccessor119.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:275)
at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
at com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:83)
at com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:206)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:647)
... 41 more
Build Log:
[...truncated 12679 lines...]
[junit4] Suite: org.apache.solr.core.TestJmxIntegration
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.core.TestJmxIntegration_F5EFC91ED1D0469-001/init-core-data-001
[junit4] 2> 2218925 WARN (SUITE-TestJmxIntegration-seed#[F5EFC91ED1D0469]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=3 numCloses=3
[junit4] 2> 2218925 INFO (SUITE-TestJmxIntegration-seed#[F5EFC91ED1D0469]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields
[junit4] 2> 2218927 INFO (SUITE-TestJmxIntegration-seed#[F5EFC91ED1D0469]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 2218928 INFO (SUITE-TestJmxIntegration-seed#[F5EFC91ED1D0469]-worker) [ ] o.a.s.SolrTestCaseJ4 ####initCore
[junit4] 2> 2218928 INFO (SUITE-TestJmxIntegration-seed#[F5EFC91ED1D0469]-worker) [ ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
[junit4] 2> 2218962 INFO (SUITE-TestJmxIntegration-seed#[F5EFC91ED1D0469]-worker) [ ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2218988 INFO (SUITE-TestJmxIntegration-seed#[F5EFC91ED1D0469]-worker) [ ] o.a.s.s.IndexSchema [null] Schema name=test
[junit4] 2> 2219105 INFO (SUITE-TestJmxIntegration-seed#[F5EFC91ED1D0469]-worker) [ ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 2219143 INFO (SUITE-TestJmxIntegration-seed#[F5EFC91ED1D0469]-worker) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@409ba446
[junit4] 2> 2219147 INFO (SUITE-TestJmxIntegration-seed#[F5EFC91ED1D0469]-worker) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@409ba446
[junit4] 2> 2219147 INFO (SUITE-TestJmxIntegration-seed#[F5EFC91ED1D0469]-worker) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@409ba446
[junit4] 2> 2219149 INFO (coreLoadExecutor-6303-thread-1) [ x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
[junit4] 2> 2219192 INFO (coreLoadExecutor-6303-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2219218 INFO (coreLoadExecutor-6303-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 2219340 INFO (coreLoadExecutor-6303-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 2219346 INFO (coreLoadExecutor-6303-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1, trusted=true
[junit4] 2> 2219346 INFO (coreLoadExecutor-6303-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@409ba446
[junit4] 2> 2219347 INFO (coreLoadExecutor-6303-thread-1) [ x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2219347 INFO (coreLoadExecutor-6303-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.core.TestJmxIntegration_F5EFC91ED1D0469-001/init-core-data-001/]
[junit4] 2> 2219348 INFO (coreLoadExecutor-6303-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=30, maxMergeAtOnceExplicit=14, maxMergedSegmentMB=48.7626953125, floorSegmentMB=1.78125, forceMergeDeletesPctAllowed=5.68314852536186, segmentsPerTier=17.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 2219406 INFO (coreLoadExecutor-6303-thread-1) [ x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2219407 INFO (coreLoadExecutor-6303-thread-1) [ x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2219407 INFO (coreLoadExecutor-6303-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2219407 INFO (coreLoadExecutor-6303-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2219408 INFO (coreLoadExecutor-6303-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=26, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 2219409 INFO (coreLoadExecutor-6303-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4434a78b[collection1] main]
[junit4] 2> 2219409 INFO (coreLoadExecutor-6303-thread-1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
[junit4] 2> 2219415 INFO (coreLoadExecutor-6303-thread-1) [ x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4] 2> 2219438 INFO (coreLoadExecutor-6303-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2219438 INFO (coreLoadExecutor-6303-thread-1) [ x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1569936474455932928
[junit4] 2> 2219439 INFO (searcherExecutor-6304-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
[junit4] 2> 2219441 INFO (searcherExecutor-6304-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
[junit4] 2> 2219441 INFO (searcherExecutor-6304-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
[junit4] 2> 2219441 INFO (searcherExecutor-6304-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
[junit4] 2> 2219441 INFO (searcherExecutor-6304-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
[junit4] 2> 2219441 INFO (searcherExecutor-6304-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
[junit4] 2> 2219442 INFO (searcherExecutor-6304-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
[junit4] 2> 2219442 INFO (searcherExecutor-6304-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
[junit4] 2> 2219442 INFO (searcherExecutor-6304-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
[junit4] 2> 2219442 INFO (SUITE-TestJmxIntegration-seed#[F5EFC91ED1D0469]-worker) [ ] o.a.s.SolrTestCaseJ4 ####initCore end
[junit4] 2> 2219444 INFO (searcherExecutor-6304-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@4434a78b[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2219445 INFO (TEST-TestJmxIntegration.testJmxUpdate-seed#[F5EFC91ED1D0469]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testJmxUpdate
[junit4] 2> 2219473 INFO (TEST-TestJmxIntegration.testJmxUpdate-seed#[F5EFC91ED1D0469]) [ ] o.a.s.c.TestJmxIntegration Mbeans in server: [solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/admin/luke,name=errors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/admin/logging,name=timeouts, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=QUERY,scope=/dump,name=timeouts, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/admin/threads,name=requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/admin/mbeans,name=clientErrors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/ping,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=QUERY,scope=/dump,name=clientErrors, solr:dom1=core,dom2=collection1,category=UPDATE,scope=update,name=requestTimes, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=QUERY,scope=/dump,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=QUERY,scope=/terms,name=requests, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/properties,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=REPLICATION,scope=/replication,name=replicateAfter, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update,name=handlerStart, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update,name=totalTime, solr:dom1=core,dom2=collection1,category=UPDATE,scope=updateHandler,name=rollbacks, solr:dom1=core,dom2=collection1,category=QUERY,scope=/search-facet-def,name=timeouts, solr:dom1=node,category=ADMIN,scope=/admin/cores,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=UPDATE,scope=updateHandler,name=cumulativeAdds, solr:dom1=node,category=ADMIN,scope=/admin/collections,name=requestTimes, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=REPLICATION,scope=peerSync,name=errors, solr:dom1=core,dom2=collection1,category=HIGHLIGHTER,scope=SolrFragmenter,name=gap.requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=UPDATE,scope=/update/json/docs,name=clientErrors, solr:dom1=node,category=ADMIN,scope=/admin/authorization,name=requestTimes, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/ping,name=errors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/schema,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=TLOG,scope=buffered,name=ops, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/admin/mbeans,name=errors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=QUERY,scope=/stream,name=requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=UPDATE,scope=updateHandler,name=commits, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=QUERY,scope=/stream,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=QUERY,scope=/sql,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=UPDATE,scope=/update/json/docs,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=REPLICATION,scope=peerSync,name=skipped, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=QUERY,scope=/dump,name=timeouts, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=CORE,name=shard, solr:dom1=core,dom2=collection1,category=QUERY,scope=/search-facet-def,name=errors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=QUERY,scope=/get,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=ADMIN,scope=/admin/threads,name=totalTime, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/properties,name=timeouts, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/schema,name=handlerStart, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=UPDATE,scope=update,name=timeouts, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=QUERY,scope=/export,name=clientErrors, solr:dom1=node,category=ADMIN,scope=/admin/authorization,name=handlerStart, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=REPLICATION,scope=/replication,name=isSlave, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update,name=clientErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/search-facet-invariants,name=clientErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/export,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=QUERY,scope=/sql,name=timeouts, solr:dom1=core,dom2=collection1,category=QUERY,scope=spellCheckWithWordbreak_Direct,name=errors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=HIGHLIGHTER,scope=SolrBoundaryScanner,name=default.requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/logging,name=requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=QUERY,scope=/debug/dump,name=requestTimes, solr:dom1=node,category=QUERY,scope=httpShardHandler,name=threadPool.httpShardExecutor.completed, solr:dom1=core,dom2=collection1,category=QUERY,scope=/debug/dump,name=clientErrors, java.lang:type=Runtime, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=SEARCHER,name=new, solr:dom1=node,category=ADMIN,scope=/admin/authorization,name=errors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=QUERY,scope=standard,name=handlerStart, solr:dom1=node,category=UPDATE,scope=updateShardHandler,name=threadPool.recoveryExecutor.duration, solr:dom1=core,dom2=collection1,category=CORE,scope=fs,name=totalSpace, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=ADMIN,scope=/config,name=clientErrors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=TLOG,scope=applyingBuffered,name=ops, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/segments,name=timeouts, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/admin/threads,name=totalTime, solr:dom1=node,category=CONTAINER,scope=cores,name=unloaded, solr:dom1=core,dom2=collection1,category=QUERY,scope=tvrh,name=errors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=QUERY,scope=/dump,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=QUERY,scope=/graph,name=timeouts, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=ADMIN,scope=/admin/ping,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/ping,name=timeouts, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=REPLICATION,scope=/replication,name=indexVersion, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=INDEX,name=size, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=UPDATE,scope=/update/csv,name=requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=ADMIN,scope=/admin/threads,name=timeouts, solr:dom1=core,dom2=collection1,category=UPDATE,scope=updateHandler,name=cumulativeErrors, solr:dom1=node,category=UPDATE,scope=updateShardHandler,name=threadPool.recoveryExecutor.running, solr:dom1=core,dom2=collection1,category=QUERY,scope=defaults,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/admin/plugins,name=requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=UPDATE,scope=updateHandler,name=cumulativeDeletesByQuery, solr:dom1=jvm,name0=memory,name1=pools,name2=Compressed-Class-Space,name=used, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/admin/system,name=requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=QUERY,scope=/debug/dump,name=timeouts, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=QUERY,scope=standard,name=requests, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/schema,name=requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=REPLICATION,scope=/replication,name=totalTime, solr:dom1=core,dom2=collection1,category=HIGHLIGHTER,scope=SolrFormatter,name=html.requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=QUERY,scope=/sql,name=handlerStart, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=INDEX,name=size, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=QUERY,scope=/get,name=serverErrors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/logging,name=serverErrors, solr:dom1=core,dom2=collection1,category=TLOG,scope=replay,name=ops, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=ADMIN,scope=/admin/segments,name=timeouts, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=QUERY,scope=/debug/dump,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=QUERY,scope=/graph,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=QUERY,scope=/get,name=timeouts, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/threads,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=ADMIN,scope=/admin/segments,name=requests, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/ping,name=requests, solr:dom1=core,dom2=collection1,category=QUERY,scope=/export,name=timeouts, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/admin/properties,name=errors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=UPDATE,scope=updateHandler,name=errors, solr:dom1=node,category=ADMIN,scope=/admin/configs,name=errors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=HIGHLIGHTER,scope=SolrFragListBuilder,name=default.requests, solr:dom1=core,dom2=collection1,category=UPDATE,scope=update,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/file,name=requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/admin/logging,name=errors, solr:dom1=core,dom2=collection1,category=QUERY,scope=defaults,name=errors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=QUERY,scope=/terms,name=requestTimes, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=UPDATE,scope=/update/csv,name=handlerStart, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=REPLICATION,scope=/replication,name=confFilesToReplicate, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=QUERY,scope=/export,name=requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=CORE,name=shard, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=ADMIN,scope=/admin/mbeans,name=requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=SEARCHER,scope=new,name=time, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=QUERY,scope=/export,name=requestTimes, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/schema,name=requestTimes, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=QUERY,scope=/debug/dump,name=timeouts, solr:dom1=core,dom2=collection1,category=QUERY,scope=spellCheckWithWordbreak_Direct,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=QUERY,scope=/graph,name=requests, solr:dom1=node,category=ADMIN,scope=/admin/zookeeper,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=QUERY,scope=/graph,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=QUERY,scope=/dump,name=errors, solr:dom1=core,dom2=collection1,category=QUERY,scope=spellCheckCompRH_Direct,name=handlerStart, solr:dom1=node,category=ADMIN,scope=/admin/configs,name=requests, solr:dom1=core,dom2=collection1,category=QUERY,scope=defaults,name=requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=TLOG,name=state, solr:dom1=core,dom2=collection1,category=QUERY,scope=/search-facet-invariants,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/admin/segments,name=requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=REPLICATION,scope=/replication,name=indexPath, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/config,name=timeouts, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=REPLICATION,scope=/replication,name=serverErrors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=REPLICATION,scope=/replication,name=indexVersion, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/segments,name=requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=UPDATE,scope=updateHandler,name=adds, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=UPDATE,scope=/update/json/docs,name=serverErrors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=CORE,name=startTime, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=QUERY,scope=/dump,name=timeouts, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=UPDATE,scope=/update/json,name=requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=HIGHLIGHTER,scope=SolrFragmentsBuilder,name=default.requests, solr:dom1=core,dom2=collection1,category=QUERY,scope=/graph,name=requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=UPDATE,scope=updateHandler,name=optimizes, solr:dom1=node,category=ADMIN,scope=/admin/cores,name=requestTimes, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=QUERY,scope=/sql,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=QUERY,scope=/get,name=handlerStart, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=UPDATE,scope=update,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/admin/properties,name=requestTimes, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/segments,name=requests, solr:dom1=core,dom2=collection1,category=QUERY,scope=/search-facet-invariants,name=timeouts, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/plugins,name=totalTime, solr:dom1=core,dom2=collection1,category=UPDATE,scope=update,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=ADMIN,scope=/admin/system,name=requestTimes, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=CORE,name=instanceDir, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=UPDATE,scope=/update/csv,name=errors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/mbeans,name=requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=CORE,name=startTime, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/properties,name=errors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=HIGHLIGHTER,scope=SolrFragmenter,name=default.requests, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=QUERY,scope=/get,name=requests, solr:dom1=jvm,name0=memory,name1=pools,name2=Metaspace,name=usage, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/plugins,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=QUERY,scope=/debug/dump,name=requestTimes, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=QUERY,scope=/sql,name=timeouts, solr:dom1=core,dom2=collection1,category=QUERY,scope=/mlt,name=errors, solr:dom1=core,dom2=control_coll
[junit4] 2> ection,dom3=shard1,dom4=core_node1,category=TLOG,scope=buffered,name=ops, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/schema,name=errors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/admin/ping,name=timeouts, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/properties,name=totalTime, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/csv,name=clientErrors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=UPDATE,scope=updateHandler,name=splits, solr:dom1=node,category=QUERY,scope=/admin/autoscaling,name=errors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=ADMIN,scope=/config,name=handlerStart, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=UPDATE,scope=update,name=requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=REPLICATION,scope=/replication,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=QUERY,scope=/export,name=errors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=QUERY,scope=/get,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=QUERY,scope=/dump,name=requestTimes, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/config,name=clientErrors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/logging,name=requests, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=REPLICATION,scope=/replication,name=timeouts, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=UPDATE,scope=/update/json/docs,name=requests, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/segments,name=requestTimes, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/config,name=errors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=ADMIN,scope=/admin/luke,name=handlerStart, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/threads,name=totalTime, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=QUERY,scope=/dump,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/ping,name=totalTime, solr:dom1=core,dom2=collection1,category=QUERY,scope=/graph,name=timeouts, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=UPDATE,scope=updateHandler,name=deletesByQuery, solr:dom1=core,dom2=collection1,category=REPLICATION,scope=/replication,name=totalTime, solr:dom1=jvm,name0=gc,name1=PS-MarkSweep,name=time, solr:dom1=node,category=ADMIN,scope=/admin/collections,name=requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/properties,name=errors, solr:dom1=node,category=ADMIN,scope=/admin/info,name=handlerStart, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/admin/segments,name=serverErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=mock,name=timeouts, solr:dom1=core,dom2=collection1,category=UPDATE,scope=update,name=handlerStart, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=SEARCHER,scope=new,name=time, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/csv,name=timeouts, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=UPDATE,scope=/update,name=errors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/luke,name=timeouts, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/threads,name=totalTime, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=QUERY,scope=/terms,name=handlerStart, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=QUERY,scope=/stream,name=handlerStart, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=TLOG,scope=replay,name=remaining.logs, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/admin/mbeans,name=totalTime, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/luke,name=requestTimes, solr:dom1=node,category=QUERY,scope=/admin/metrics/collector,name=timeouts, solr:dom1=core,dom2=collection1,category=QUERY,scope=spellCheckWithWordbreak_Direct,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=ADMIN,scope=/admin/plugins,name=timeouts, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/segments,name=serverErrors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=UPDATE,scope=/update/csv,name=errors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/luke,name=serverErrors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=CORE,name=collection, solr:dom1=node,category=QUERY,scope=/admin/autoscaling,name=requests, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=QUERY,scope=/sql,name=errors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/logging,name=timeouts, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=UPDATE,scope=/update,name=errors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/config,name=totalTime, solr:dom1=node,category=ADMIN,scope=/admin/info,name=requestTimes, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/threads,name=requests, solr:dom1=node,category=ADMIN,scope=/admin/authorization,name=clientErrors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=UPDATE,scope=updateHandler,name=merges, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=QUERY,scope=/stream,name=timeouts, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=QUERY,scope=/dump,name=requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/mbeans,name=errors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=ADMIN,scope=/schema,name=timeouts, solr:dom1=jvm,name0=os,name=freeSwapSpaceSize, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=QUERY,scope=/debug/dump,name=errors, solr:dom1=core,dom2=collection1,category=REPLICATION,scope=/replication,name=indexPath, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=UPDATE,scope=updateHandler,name=docsPending, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/threads,name=handlerStart, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=QUERY,scope=/terms,name=clientErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/export,name=requests, solr:dom1=core,dom2=collection1,category=REPLICATION,scope=/replication,name=requestTimes, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=QUERY,scope=/sql,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=QUERY,scope=/graph,name=errors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/admin/luke,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=UPDATE,scope=/update/json,name=errors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=REPLICATION,scope=/replication,name=requests, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/file,name=timeouts, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/properties,name=clientErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=spellCheckCompRH_Direct,name=totalTime, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=CORE,name=refCount, java.nio:type=BufferPool,name=mapped, com.sun.management:type=DiagnosticCommand, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/file,name=clientErrors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/plugins,name=handlerStart, solr:dom1=core,dom2=collection1,category=SEARCHER,scope=new,name=maxReached, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/plugins,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=ADMIN,scope=/admin/segments,name=handlerStart, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=QUERY,scope=/export,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=UPDATE,scope=/update/csv,name=totalTime, solr:dom1=node,category=UPDATE,scope=updateShardHandler,name=threadPool.updateExecutor.duration, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/csv,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=ADMIN,scope=/admin/threads,name=requestTimes, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=ADMIN,scope=/admin/plugins,name=requestTimes, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/mbeans,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/admin/file,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/system,name=handlerStart, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=REPLICATION,scope=/replication,name=requestTimes, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/threads,name=handlerStart, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=INDEX,name=sizeInBytes, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/config,name=totalTime, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/schema,name=timeouts, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/config,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=REPLICATION,scope=/replication,name=isSlave, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=QUERY,scope=/stream,name=totalTime, solr:dom1=core,dom2=collection1,category=QUERY,scope=spellCheckCompRH,name=errors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=ADMIN,scope=/admin/file,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=REPLICATION,scope=peerSync,name=time, solr:dom1=core,dom2=collection1,category=QUERY,scope=dismax,name=timeouts, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=UPDATE,scope=/update/json,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=ADMIN,scope=/admin/luke,name=requestTimes, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/plugins,name=errors, solr:dom1=jvm,name0=threads,name1=timed_waiting,name=count, solr:dom1=core,dom2=collection1,category=CORE,name=instanceDir, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=REPLICATION,scope=/replication,name=handlerStart, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=QUERY,scope=/graph,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=ADMIN,scope=/admin/luke,name=serverErrors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/file,name=totalTime, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/plugins,name=handlerStart, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/logging,name=clientErrors, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/json,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=QUERY,scope=/stream,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/segments,name=handlerStart, solr:dom1=jvm,name0=memory,name1=pools,name2=Metaspace,name=used, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=UPDATE,scope=/update/csv,name=requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/admin/luke,name=requestTimes, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/config,name=requests, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=QUERY,scope=/graph,name=timeouts, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=QUERY,scope=/stream,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=UPDATE,scope=updateHandler,name=cumulativeAdds, solr:dom1=core,dom2=collection1,category=QUERY,scope=spellCheckCompRH,name=handlerStart, solr:dom1=node,category=ADMIN,scope=/admin/collections,name=totalTime, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/plugins,name=requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=UPDATE,scope=/update/json,name=clientErrors, solr:dom1=node,category=CONTAINER,scope=threadPool,name=coreContainerWorkExecutor.submitted, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=QUERY,scope=/stream,name=requests, solr:dom1=core,dom2=collection1,category=QUERY,scope=spellCheckWithWordbreak_Direct,name=handlerStart, solr:dom1=core,dom2=collection1,category=CACHE,scope=core,name=fieldCache, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=UPDATE,scope=updateHandler,name=cumulativeDeletesById, solr:dom1=jvm,name0=memory,name1=pools,name2=Code-Cache,name=max, solr:dom1=core,dom2=collection1,category=QUERY,scope=spellCheckCompRH,name=requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=UPDATE,scope=updateHandler,name=errors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=REPLICATION,scope=/replication,name=isMaster, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=UPDATE,scope=updateHandler,name=rollbacks, solr:dom1=jvm,name0=memory,name1=pools,name2=PS-Survivor-Space,name=committed, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=UPDATE,scope=updateHandler,name=softAutoCommits, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=TLOG,name=state, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=SEARCHER,scope=new,name=errors, solr:dom1=core,dom2=collection1,category=HIGHLIGHTER,scope=SolrFragmenter,name=default.requests, solr:dom1=node,category=ADMIN,scope=/admin/info,name=requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=UPDATE,scope=/update/json,name=serverErrors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/threads,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/segments,name=errors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/config,name=errors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/file,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=ADMIN,scope=/schema,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=QUERY,scope=/stream,name=errors, solr:dom1=core,dom2=collection1,category=UPDATE,scope=updateHandler,name=docsPending, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=UPDATE,scope=/update/csv,name=serverErrors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=QUERY,scope=standard,name=timeouts, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/luke,name=errors, solr:dom1=jvm,name0=memory,name1=pools,name2=PS-Survivor-Space,name=used-after-gc, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/config,name=handlerStart, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=QUERY,scope=/graph,name=clientErrors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=QUERY,scope=/stream,name=clientErrors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,ca
[junit4] 2> tegory=ADMIN,scope=/admin/segments,name=totalTime, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=QUERY,scope=/sql,name=clientErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/export,name=clientErrors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/logging,name=requestTimes, solr:dom1=jvm,name0=memory,name1=total,name=max, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=QUERY,scope=standard,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=UPDATE,scope=/update,name=requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=ADMIN,scope=/admin/file,name=timeouts, solr:dom1=core,dom2=collection1,category=QUERY,scope=/debug/dump,name=errors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=ADMIN,scope=/admin/system,name=serverErrors, solr:dom1=core,dom2=collection1,category=SEARCHER,scope=searcher,name=registeredAt, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/admin/logging,name=serverErrors, solr:dom1=node,category=ADMIN,scope=/admin/info,name=serverErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/export,name=handlerStart, solr:dom1=jvm,name0=memory,name1=pools,name2=PS-Survivor-Space,name=used, solr:dom1=core,dom2=collection1,category=CORE,name=startTime, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/admin/properties,name=serverErrors, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/json/docs,name=requestTimes, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=ADMIN,scope=/schema,name=requestTimes, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=QUERY,scope=/sql,name=errors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=QUERY,scope=standard,name=requestTimes, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/system,name=totalTime, solr:dom1=core,dom2=collection1,category=QUERY,scope=spellCheckCompRH1,name=serverErrors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/ping,name=requestTimes, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=UPDATE,scope=/update/json,name=requestTimes, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=ADMIN,scope=/admin/properties,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=QUERY,scope=/debug/dump,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/segments,name=timeouts, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=UPDATE,scope=/update,name=errors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=QUERY,scope=/export,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=UPDATE,scope=/update/json,name=requests, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=QUERY,scope=/debug/dump,name=serverErrors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/file,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/properties,name=handlerStart, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=QUERY,scope=/terms,name=timeouts, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=QUERY,scope=/stream,name=requestTimes, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=QUERY,scope=/debug/dump,name=requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=UPDATE,scope=/update/csv,name=handlerStart, solr:dom1=core,dom2=collection1,category=QUERY,scope=mock,name=serverErrors, solr:dom1=jvm,name0=buffers,name1=mapped,name=MemoryUsed, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=UPDATE,scope=/update/json/docs,name=requestTimes, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/admin/system,name=serverErrors, solr:dom1=jvm,name0=buffers,name1=direct,name=Count, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/plugins,name=serverErrors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/mbeans,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=QUERY,scope=/dump,name=clientErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=mock,name=requestTimes, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/plugins,name=serverErrors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=UPDATE,scope=updateHandler,name=cumulativeErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=mock,name=totalTime, solr:dom1=jvm,name0=memory,name1=heap,name=committed, solr:dom1=jvm,name0=threads,name1=deadlock,name=count, solr:dom1=core,dom2=collection1,category=SEARCHER,scope=searcher,name=warmupTime, solr:dom1=jvm,name0=threads,name1=runnable,name=count, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=QUERY,scope=/dump,name=errors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=QUERY,scope=/dump,name=requestTimes, solr:dom1=node,category=ADMIN,scope=/admin/metrics,name=clientErrors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=UPDATE,scope=updateHandler,name=autoCommitMaxTime, solr:dom1=jvm,name0=memory,name1=heap,name=init, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=UPDATE,scope=/update,name=totalTime, solr:dom1=core,dom2=collection1,category=HIGHLIGHTER,scope=SolrFormatter,name=default.requests, solr:dom1=node,category=ADMIN,scope=/admin/configs,name=requestTimes, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/config,name=requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=UPDATE,scope=updateHandler,name=autoCommitMaxTime, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=SEARCHER,scope=new,name=maxReached, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/ping,name=errors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=QUERY,scope=/debug/dump,name=handlerStart, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/admin/logging,name=requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=UPDATE,scope=/update,name=totalTime, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/system,name=requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=QUERY,scope=standard,name=clientErrors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/luke,name=handlerStart, solr:dom1=node,category=CONTAINER,scope=cores,name=lazy, solr:dom1=core,dom2=collection1,category=QUERY,scope=/search-facet-invariants,name=handlerStart, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=REPLICATION,scope=/replication,name=replicationEnabled, solr:dom1=core,dom2=collection1,category=QUERY,scope=/search-facet-invariants,name=requests, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/properties,name=errors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/logging,name=requests, solr:dom1=jvm,name0=memory,name1=pools,name2=PS-Eden-Space,name=init, solr:dom1=jvm,name0=memory,name1=pools,name2=Code-Cache,name=committed, solr:dom1=node,category=QUERY,scope=httpShardHandler,name=pendingConnections, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=REPLICATION,scope=/replication,name=indexPath, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/threads,name=errors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=QUERY,scope=standard,name=timeouts, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=ADMIN,scope=/admin/properties,name=errors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/admin/segments,name=timeouts, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=QUERY,scope=/terms,name=clientErrors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=QUERY,scope=standard,name=errors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=UPDATE,scope=updateHandler,name=cumulativeAdds, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=QUERY,scope=/export,name=requests, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/schema,name=requestTimes, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/segments,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/luke,name=requestTimes, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/mbeans,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=ADMIN,scope=/admin/mbeans,name=clientErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/stream,name=serverErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=spellCheckCompRH_Direct,name=timeouts, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=QUERY,scope=/get,name=timeouts, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=QUERY,scope=/terms,name=timeouts, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=QUERY,scope=/export,name=timeouts, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/schema,name=serverErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=dismax,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=QUERY,scope=standard,name=errors, solr:dom1=node,category=ADMIN,scope=/admin/info,name=errors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/search-facet-def,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/mbeans,name=requestTimes, solr:dom1=core,dom2=collection1,category=QUERY,scope=/terms,name=serverErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=dismax,name=requests, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=QUERY,scope=/stream,name=timeouts, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=UPDATE,scope=/update/json,name=handlerStart, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=QUERY,scope=/stream,name=errors, solr:dom1=jvm,name0=os,name=systemLoadAverage, solr:dom1=core,dom2=collection1,category=QUERY,scope=/search-facet-def,name=requestTimes, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=QUERY,scope=standard,name=errors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/admin/plugins,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=ADMIN,scope=/admin/luke,name=requests, solr:dom1=core,dom2=collection1,category=QUERY,scope=spellCheckWithWordbreak,name=clientErrors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=QUERY,scope=/graph,name=requests, solr:dom1=core,dom2=collection1,category=QUERY,scope=standard,name=errors, solr:dom1=node,category=UPDATE,scope=updateShardHandler,name=threadPool.recoveryExecutor.submitted, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=UPDATE,scope=updateHandler,name=autoCommitMaxTime, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/json,name=timeouts, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/segments,name=requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/schema,name=clientErrors, solr:dom1=core,dom2=collection1,category=SEARCHER,scope=searcher,name=reader, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=CORE,name=coreName, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/admin/logging,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=UPDATE,scope=update,name=clientErrors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/segments,name=timeouts, solr:dom1=core,dom2=collection1,category=HIGHLIGHTER,scope=SolrBoundaryScanner,name=breakIterator.requests, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/segments,name=errors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=CORE,scope=fs,name=totalSpace, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=REPLICATION,scope=/replication,name=indexVersion, solr:dom1=core,dom2=collection1,category=QUERY,scope=tvrh,name=requestTimes, solr:dom1=jvm,name0=memory,name1=pools,name2=PS-Survivor-Space,name=usage, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=CORE,name=aliases, solr:dom1=jvm,name0=memory,name1=pools,name2=PS-Old-Gen,name=init, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/admin/system,name=timeouts, solr:dom1=core,dom2=collection1,category=QUERY,scope=/stream,name=timeouts, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=QUERY,scope=/terms,name=errors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=UPDATE,scope=update,name=handlerStart, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/system,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/config,name=handlerStart, solr:dom1=core,dom2=collection1,category=UPDATE,scope=updateHandler,name=errors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/sql,name=requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=QUERY,scope=/debug/dump,name=serverErrors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=UPDATE,scope=/update/json/docs,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=SEARCHER,scope=new,name=errors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=SEARCHER,scope=new,name=warmup, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=QUERY,scope=standard,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/logging,name=timeouts, solr:dom1=core,dom2=collection1,category=QUERY,scope=defaults,name=requestTimes, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=REPLICATION,scope=peerSync,name=time, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=ADMIN,scope=/schema,name=totalTime, solr:dom1=core,dom2=collection1,category=TLOG,scope=replay,name=remaining.bytes, solr:dom1=core,dom2=collection1,category=QUERY,scope=tvrh,name=timeouts, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=UPDATE,scope=/update/json/docs,name=errors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=QUERY,scope=/dump,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=INDEX,name=size, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=UPDATE,scope=update,name=requestTimes, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/admin/plugins,name=errors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/admin/file,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/ping,name=requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/plugins,name=requests, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/csv,name=handlerStart, solr:dom1=core,dom2=collection1,category=REPLICATION,scope=/replication,name=handlerStart, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=QUERY,scope=/get,name=totalTime, solr:dom1=core,dom2=collection1,category=UPDATE,scope=updat
[junit4] 2> eHandler,name=cumulativeDeletesById, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=REPLICATION,scope=/replication,name=generation, solr:dom1=core,dom2=collection1,category=UPDATE,scope=updateHandler,name=cumulativeDeletesByQuery, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=QUERY,scope=/sql,name=handlerStart, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/admin/mbeans,name=timeouts, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=QUERY,scope=/get,name=totalTime, solr:dom1=core,dom2=collection1,category=QUERY,scope=/terms,name=clientErrors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/mbeans,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=ADMIN,scope=/admin/system,name=timeouts, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=UPDATE,scope=updateHandler,name=autoCommits, solr:dom1=jvm,name0=os,name=totalPhysicalMemorySize, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=UPDATE,scope=/update/json,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=QUERY,scope=/get,name=totalTime, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=REPLICATION,scope=/replication,name=replicationEnabled, solr:dom1=jvm,name0=memory,name1=pools,name2=Code-Cache,name=init, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=ADMIN,scope=/admin/file,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=REPLICATION,scope=/replication,name=isSlave, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=QUERY,scope=/dump,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=REPLICATION,scope=/replication,name=requestTimes, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=QUERY,scope=/terms,name=handlerStart, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=QUERY,scope=standard,name=requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/schema,name=serverErrors, solr:dom1=node,category=ADMIN,scope=/admin/zookeeper,name=timeouts, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/system,name=requestTimes, solr:dom1=node,category=ADMIN,scope=/admin/authorization,name=timeouts, solr:dom1=jvm,name0=memory,name1=pools,name2=Code-Cache,name=usage, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=TLOG,scope=applyingBuffered,name=ops, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=UPDATE,scope=/update,name=serverErrors, solr:dom1=core,dom2=collection1,category=UPDATE,scope=updateHandler,name=deletesByQuery, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/json/docs,name=serverErrors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=QUERY,scope=/dump,name=errors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/mbeans,name=requestTimes, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=QUERY,scope=/stream,name=errors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=TLOG,scope=replay,name=remaining.bytes, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=QUERY,scope=/terms,name=handlerStart, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/luke,name=totalTime, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=TLOG,scope=replay,name=remaining.logs, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/logging,name=serverErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=spellCheckCompRH1,name=clientErrors, solr:dom1=jvm,name0=memory,name1=pools,name2=PS-Old-Gen,name=committed, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=QUERY,scope=/get,name=requestTimes, solr:dom1=core,dom2=collection1,category=SEARCHER,scope=searcher,name=openedAt, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=UPDATE,scope=/update/json,name=requests, solr:dom1=core,dom2=collection1,category=QUERY,scope=/mlt,name=requestTimes, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/mbeans,name=serverErrors, solr:dom1=core,dom2=collection1,category=UPDATE,scope=update,name=errors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=QUERY,scope=/get,name=requests, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=UPDATE,scope=updateHandler,name=errors, solr:dom1=node,category=QUERY,scope=httpShardHandler,name=leasedConnections, solr:dom1=core,dom2=collection1,category=UPDATE,scope=updateHandler,name=softAutoCommits, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=UPDATE,scope=/update,name=handlerStart, solr:dom1=jvm,name0=memory,name1=non-heap,name=used, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/logging,name=clientErrors, java.lang:type=MemoryPool,name=Code Cache, java.nio:type=BufferPool,name=direct, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/system,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=UPDATE,scope=updateHandler,name=optimizes, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=ADMIN,scope=/admin/segments,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=UPDATE,scope=/update/json,name=clientErrors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=UPDATE,scope=/update/json/docs,name=timeouts, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=ADMIN,scope=/admin/mbeans,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=UPDATE,scope=update,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=QUERY,scope=/graph,name=totalTime, solr:dom1=core,dom2=collection1,category=REPLICATION,scope=/replication,name=serverErrors, solr:dom1=core,dom2=collection1,category=UPDATE,scope=update,name=timeouts, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/luke,name=errors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/admin/file,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=UPDATE,scope=/update,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=UPDATE,scope=/update/json/docs,name=clientErrors, solr:dom1=core,dom2=collection1,category=REPLICATION,scope=/replication,name=indexSize, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=ADMIN,scope=/admin/luke,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=UPDATE,scope=/update/json,name=timeouts, solr:dom1=jvm,name0=buffers,name1=mapped,name=Count, java.lang:type=Compilation, solr:dom1=node,category=UPDATE,scope=updateShardHandler,name=threadPool.updateExecutor.submitted, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/file,name=handlerStart, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=QUERY,scope=/sql,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=CORE,name=refCount, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/threads,name=handlerStart, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=QUERY,scope=/dump,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=ADMIN,scope=/admin/luke,name=errors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=REPLICATION,scope=/replication,name=timeouts, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=QUERY,scope=/graph,name=requestTimes, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=QUERY,scope=/debug/dump,name=requests, solr:dom1=core,dom2=collection1,category=CACHE,scope=searcher,name=documentCache, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/mbeans,name=handlerStart, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=QUERY,scope=/export,name=timeouts, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=QUERY,scope=/sql,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=QUERY,scope=/stream,name=requestTimes, java.lang:type=MemoryPool,name=Compressed Class Space, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=HIGHLIGHTER,scope=SolrFormatter,name=default.requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=QUERY,scope=/dump,name=totalTime, solr:dom1=jvm,name0=os,name=committedVirtualMemorySize, solr:dom1=jvm,name0=memory,name1=non-heap,name=max, solr:dom1=core,dom2=collection1,category=CORE,name=coreName, solr:dom1=node,category=CONTAINER,scope=threadPool,name=coreLoadExecutor.running, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=ADMIN,scope=/config,name=errors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/sql,name=handlerStart, solr:dom1=jvm,name0=memory,name1=heap,name=used, solr:dom1=core,dom2=collection1,category=QUERY,scope=standard,name=requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/admin/threads,name=timeouts, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=REPLICATION,scope=/replication,name=replicateAfter, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/threads,name=timeouts, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/json/docs,name=requests, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/system,name=requestTimes, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=ADMIN,scope=/admin/mbeans,name=handlerStart, solr:dom1=core,dom2=collection1,category=QUERY,scope=/terms,name=handlerStart, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=ADMIN,scope=/config,name=requestTimes, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=QUERY,scope=/get,name=errors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/system,name=handlerStart, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=QUERY,scope=/graph,name=requests, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=TLOG,name=state, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=ADMIN,scope=/admin/plugins,name=errors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=UPDATE,scope=/update/csv,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=QUERY,scope=/sql,name=requestTimes, solr:dom1=node,category=ADMIN,scope=/admin/zookeeper,name=requestTimes, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/file,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=QUERY,scope=/terms,name=requestTimes, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=UPDATE,scope=updateHandler,name=commits, solr:dom1=core,dom2=collection1,category=QUERY,scope=spellCheckCompRH,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/luke,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/system,name=errors, java.lang:type=OperatingSystem, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=TLOG,scope=replay,name=remaining.bytes, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=ADMIN,scope=/admin/properties,name=timeouts, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/admin/mbeans,name=requestTimes, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=UPDATE,scope=updateHandler,name=softAutoCommits, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=UPDATE,scope=/update/json,name=handlerStart, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/json/docs,name=timeouts, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/mbeans,name=handlerStart, solr:dom1=node,category=QUERY,scope=/admin/autoscaling,name=requestTimes, solr:dom1=core,dom2=collection1,category=TLOG,scope=replay,name=remaining.logs, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=ADMIN,scope=/admin/system,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=ADMIN,scope=/config,name=serverErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/sql,name=serverErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/sql,name=totalTime, solr:dom1=node,category=ADMIN,scope=/admin/cores,name=threadPool.parallelCoreAdminExecutor.duration, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/mbeans,name=totalTime, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=UPDATE,scope=/update/json/docs,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=QUERY,scope=/stream,name=clientErrors, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/json/docs,name=totalTime, solr:dom1=jvm,name0=memory,name1=pools,name2=PS-Eden-Space,name=max, solr:dom1=jvm,name0=memory,name1=pools,name2=PS-Old-Gen,name=used, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=QUERY,scope=/debug/dump,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=QUERY,scope=/graph,name=totalTime, solr:dom1=core,dom2=collection1,category=UPDATE,scope=updateHandler,name=cumulativeAdds, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=QUERY,scope=/get,name=handlerStart, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=QUERY,scope=/export,name=totalTime, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/logging,name=timeouts, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=ADMIN,scope=/admin/threads,name=clientErrors, java.lang:type=MemoryManager,name=CodeCacheManager, solr:dom1=core,dom2=collection1,category=QUERY,scope=/get,name=totalTime, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=QUERY,scope=/terms,name=totalTime, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/file,name=requests, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=UPDATE,scope=updateHandler,name=softAutoCommits, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/schema,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=HIGHLIGHTER,scope=SolrBoundaryScanner,name=default.requests, solr:dom1=core,dom2=collection1,category=QUERY,scope=/debug/dump,name=totalTime, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/config,name=timeouts, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=QUERY,scope=/sql,name=totalTime, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=UPDATE,scope=/update/json,name=clientErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=mltrh,name=requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=UPDATE,scope=updateHandler,name=autoCommits, solr:dom1=core,dom2=collection1,category=INDEX,name=size, solr:dom1=core,dom2=collection1,category=QUERY,scope=spellCheckCompRH1,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=REPLICATION,scope=/replication,name=indexSize, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/admin/luke,name=timeouts, solr:dom1=core,dom2=collection1,category=QUERY,scope=mltrh,name
[junit4] 2> =handlerStart, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=QUERY,scope=/get,name=requestTimes, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=UPDATE,scope=updateHandler,name=softAutoCommitMaxTime, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/admin/ping,name=clientErrors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=QUERY,scope=/export,name=requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=UPDATE,scope=updateHandler,name=splits, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=QUERY,scope=/sql,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=CORE,scope=fs,name=usableSpace, solr:dom1=core,dom2=collection1,category=QUERY,scope=spellCheckWithWordbreak_Direct,name=requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=CORE,name=shard, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/config,name=requests, solr:dom1=jvm,name0=os,name=processCpuLoad, solr:dom1=jvm,name0=memory,name1=pools,name2=Metaspace,name=committed, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=QUERY,scope=/export,name=timeouts, solr:dom1=core,dom2=collection1,category=QUERY,scope=spellCheckCompRH1,name=errors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=ADMIN,scope=/admin/threads,name=handlerStart, solr:dom1=core,dom2=collection1,category=QUERY,scope=dismax,name=errors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=QUERY,scope=/get,name=serverErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=spellCheckCompRH_Direct,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=QUERY,scope=/sql,name=requestTimes, solr:dom1=node,category=QUERY,scope=/admin/metrics/collector,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=QUERY,scope=/terms,name=serverErrors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/ping,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/threads,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=UPDATE,scope=updateHandler,name=cumulativeDeletesById, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=UPDATE,scope=updateHandler,name=cumulativeErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=CORE,name=collection, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=QUERY,scope=/export,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=HIGHLIGHTER,scope=SolrFormatter,name=default.requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=QUERY,scope=/sql,name=requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=QUERY,scope=/dump,name=requests, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/ping,name=requestTimes, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=UPDATE,scope=/update/json,name=totalTime, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=QUERY,scope=/graph,name=requestTimes, solr:dom1=node,category=ADMIN,scope=/admin/info,name=timeouts, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/system,name=handlerStart, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/threads,name=requestTimes, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=UPDATE,scope=updateHandler,name=rollbacks, JMImplementation:type=MBeanServerDelegate, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=REPLICATION,scope=peerSync,name=errors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/admin/system,name=requestTimes, solr:dom1=core,dom2=collection1,category=UPDATE,scope=updateHandler,name=deletesById, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/threads,name=clientErrors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/mbeans,name=errors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/segments,name=serverErrors, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/file,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/segments,name=clientErrors, java.lang:type=MemoryPool,name=Metaspace, solr:dom1=core,dom2=collection1,category=REPLICATION,scope=/replication,name=clientErrors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=UPDATE,scope=/update/json,name=serverErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=mltrh,name=clientErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/debug/dump,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=UPDATE,scope=/update/csv,name=requestTimes, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=HIGHLIGHTER,scope=SolrFragListBuilder,name=default.requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=UPDATE,scope=/update/csv,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/plugins,name=errors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=UPDATE,scope=/update,name=handlerStart, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/plugins,name=requests, solr:dom1=core,dom2=collection1,category=CACHE,scope=searcher,name=fieldValueCache, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/mbeans,name=requests, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/logging,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/config,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=UPDATE,scope=/update/json,name=timeouts, solr:dom1=core,dom2=collection1,category=UPDATE,scope=/update/csv,name=requestTimes, solr:dom1=node,category=ADMIN,scope=/admin/cores,name=timeouts, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/segments,name=handlerStart, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/threads,name=requestTimes, solr:dom1=core,dom2=collection1,category=QUERY,scope=spellCheckWithWordbreak_Direct,name=clientErrors, solr:dom1=node,category=CONTAINER,scope=fs,name=totalSpace, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/schema,name=errors, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/schema,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node3,category=QUERY,scope=/get,name=clientErrors, solr:dom1=node,category=ADMIN,scope=/admin/configs,name=timeouts, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=UPDATE,scope=updateHandler,name=docsPending, solr:dom1=node,category=ADMIN,scope=/admin/cores,name=threadPool.parallelCoreAdminExecutor.completed, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=REPLICATION,scope=/replication,name=timeouts, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=UPDATE,scope=update,name=requestTimes, solr:dom1=core,dom2=control_collection,dom3=shard1,dom4=core_node1,category=QUERY,scope=/terms,name=serverErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=ADMIN,scope=/admin/file,name=requests, solr:dom1=core,dom2=collection1,category=SEARCHER,scope=new,name=warmup, solr:dom1=core,dom2=collection1,category=QUERY,scope=/mlt,name=clientErrors, solr:dom1=core,dom2=collection1,category=QUERY,scope=/get,name=requests, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=QUERY,scope=/terms,name=requests, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/schema,name=totalTime, solr:dom1=core,dom2=collection1,category=QUERY,scope=mltrh,name=requestTimes, solr:dom1=node,category=ADMIN,scope=/admin/zookeeper,name=totalTime, solr:dom1=jvm,name0=system,name=properties, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node2,category=QUERY,scope=/debug/dump,name=clientErrors, solr:dom1=core,dom2=collection1,dom3=shard1,dom4=core_node1,category=ADMIN,scope=/admin/properties,name=totalTime, solr:dom1=core,dom2=collection1,category=ADMIN,scope=/admin/logging,name=handlerStart, solr:dom1=core,dom2=collection1,category=QUERY,scope=sp
[...truncated too long message...]
it=} 0 20
[junit4] 2> 2219527 INFO (TEST-TestJmxIntegration.testJmxUpdate-seed#[F5EFC91ED1D0469]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testJmxUpdate
[junit4] 2> 2219532 INFO (TEST-TestJmxIntegration.testJmxRegistration-seed#[F5EFC91ED1D0469]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testJmxRegistration
[junit4] 2> 2219595 INFO (TEST-TestJmxIntegration.testJmxRegistration-seed#[F5EFC91ED1D0469]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testJmxRegistration
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestJmxIntegration -Dtests.method=testJmxRegistration -Dtests.seed=F5EFC91ED1D0469 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=es-PR -Dtests.timezone=Antarctica/Troll -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.07s J2 | TestJmxIntegration.testJmxRegistration <<<
[junit4] > Throwable #1: javax.management.RuntimeMBeanException: org.apache.lucene.store.AlreadyClosedException: this Directory is closed
[junit4] > at __randomizedtesting.SeedInfo.seed([F5EFC91ED1D0469:818F98AB805C5C0C]:0)
[junit4] > at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrow(DefaultMBeanServerInterceptor.java:839)
[junit4] > at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrowMaybeMBeanException(DefaultMBeanServerInterceptor.java:852)
[junit4] > at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:651)
[junit4] > at com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678)
[junit4] > at org.apache.solr.core.TestJmxIntegration.testJmxRegistration(TestJmxIntegration.java:121)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > Caused by: org.apache.lucene.store.AlreadyClosedException: this Directory is closed
[junit4] > at org.apache.lucene.store.BaseDirectory.ensureOpen(BaseDirectory.java:51)
[junit4] > at org.apache.lucene.store.FSDirectory.listAll(FSDirectory.java:233)
[junit4] > at org.apache.solr.core.DirectoryFactory.sizeOfDirectory(DirectoryFactory.java:264)
[junit4] > at org.apache.solr.core.SolrCore.getIndexSize(SolrCore.java:419)
[junit4] > at org.apache.solr.handler.ReplicationHandler.lambda$initializeMetrics$2(ReplicationHandler.java:867)
[junit4] > at com.codahale.metrics.JmxReporter$JmxGauge.getValue(JmxReporter.java:188)
[junit4] > at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
[junit4] > at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
[junit4] > at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
[junit4] > at com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:83)
[junit4] > at com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:206)
[junit4] > at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:647)
[junit4] > ... 41 more
[junit4] 2> 2219601 INFO (SUITE-TestJmxIntegration-seed#[F5EFC91ED1D0469]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> 2219601 INFO (SUITE-TestJmxIntegration-seed#[F5EFC91ED1D0469]-worker) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=2069680008
[junit4] 2> 2219601 INFO (SUITE-TestJmxIntegration-seed#[F5EFC91ED1D0469]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 2219603 INFO (SUITE-TestJmxIntegration-seed#[F5EFC91ED1D0469]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 2219604 INFO (SUITE-TestJmxIntegration-seed#[F5EFC91ED1D0469]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 2219604 INFO (SUITE-TestJmxIntegration-seed#[F5EFC91ED1D0469]-worker) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2219604 INFO (coreCloseExecutor-6309-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@636e24ac
[junit4] 2> 2219605 INFO (coreCloseExecutor-6309-thread-1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=1668162732
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.core.TestJmxIntegration_F5EFC91ED1D0469-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {range_facet_l_dv=PostingsFormat(name=Memory), multiDefault=PostingsFormat(name=Memory), intDefault=Lucene50(blocksize=128), id=PostingsFormat(name=Memory), range_facet_i_dv=PostingsFormat(name=Memory), range_facet_l=PostingsFormat(name=Memory), timestamp=PostingsFormat(name=Memory)}, docValues:{range_facet_l_dv=DocValuesFormat(name=Lucene70), _version_=DocValuesFormat(name=Asserting), range_facet_i_dv=DocValuesFormat(name=Memory), intDvoDefault=DocValuesFormat(name=Lucene70), timestamp=DocValuesFormat(name=Memory)}, maxPointsInLeafNode=1235, maxMBSortInHeap=5.028309977228944, sim=RandomSimilarity(queryNorm=true): {}, locale=es-PR, timezone=Antarctica/Troll
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_121 (64-bit)/cpus=4,threads=1,free=214126384,total=531628032
[junit4] 2> NOTE: All tests run in this JVM: [FacetPivotSmallTest, SolrCloudExampleTest, SolrJmxReporterTest, TestSlowCompositeReaderWrapper, CircularListTest, TestReplicationHandlerBackup, SaslZkACLProviderTest, TestConfig, TestSolrConfigHandlerConcurrent, TestScoreJoinQPNoScore, TestSolrDeletionPolicy1, TestManagedSchemaThreadSafety, CleanupOldIndexTest, HdfsRestartWhileUpdatingTest, InfoHandlerTest, TestSha256AuthenticationProvider, CollectionStateFormat2Test, ShufflingReplicaListTransformerTest, RulesTest, TestConfigSetsAPIExclusivity, ExplicitHLLTest, BadIndexSchemaTest, LukeRequestHandlerTest, TestBlobHandler, TestJsonFacetRefinement, TestFieldTypeResource, TestCorePropertiesReload, ChangedSchemaMergeTest, DistributedFacetPivotSmallAdvancedTest, TestCloudPivotFacet, StatsComponentTest, TestChildDocTransformer, TestUtils, OverseerModifyCollectionTest, SimpleFacetsTest, ZkStateWriterTest, CoreAdminHandlerTest, TestSmileRequest, CurrencyFieldXmlFileTest, HdfsTlogReplayBufferedWhileIndexingTest, TestHttpShardHandlerFactory, ClusterStateTest, TestComplexPhraseLeadingWildcard, TestFastOutputStream, OutputWriterTest, EchoParamsTest, TestInPlaceUpdatesStandalone, TestSolrCloudWithHadoopAuthPlugin, RecoveryZkTest, TestZkChroot, DocValuesMultiTest, TemplateUpdateProcessorTest, TestSubQueryTransformerDistrib, OutOfBoxZkACLAndCredentialsProvidersTest, FieldAnalysisRequestHandlerTest, TestSQLHandler, DistributedFacetExistsSmallTest, MBeansHandlerTest, BigEndianAscendingWordDeserializerTest, SolrPluginUtilsTest, UpdateParamsTest, HighlighterConfigTest, TestHighFrequencyDictionaryFactory, BinaryUpdateRequestHandlerTest, HardAutoCommitTest, UnloadDistributedZkTest, DirectUpdateHandlerOptimizeTest, PeerSyncWithIndexFingerprintCachingTest, TestRebalanceLeaders, TestJettySolrRunner, TestShortCircuitedRequests, TestSort, TestFieldCacheWithThreads, TestImplicitCoreProperties, DirectUpdateHandlerTest, TestCopyFieldCollectionResource, AutoCommitTest, RequestHandlersTest, TestCoreAdminApis, SuggesterTSTTest, BasicDistributedZk2Test, SecurityConfHandlerTest, QueryResultKeyTest, TestDistributedMissingSort, StatelessScriptUpdateProcessorFactoryTest, TestPullReplicaErrorHandling, TestOrdValues, LargeFieldTest, TestLegacyNumericUtils, TestExclusionRuleCollectionAccess, TestWriterPerf, ConjunctionSolrSpellCheckerTest, ChaosMonkeyNothingIsSafeWithPullReplicasTest, MoveReplicaTest, DistributedQueueTest, TestRestManager, TestExtendedDismaxParser, TestDFISimilarityFactory, TestLMDirichletSimilarityFactory, SpatialRPTFieldTypeTest, TestSolrCloudSnapshots, TestCustomDocTransformer, TestMiniSolrCloudClusterSSL, AddSchemaFieldsUpdateProcessorFactoryTest, DeleteReplicaTest, SolrCoreMetricManagerTest, TestTestInjection, SynonymTokenizerTest, TestPhraseSuggestions, FullSolrCloudDistribCmdsTest, TestExactSharedStatsCache, TestObjectReleaseTracker, TestApiFramework, TestIBSimilarityFactory, TestCSVResponseWriter, SuggestComponentTest, CollectionsAPISolrJTest, LeaderElectionIntegrationTest, ZkCLITest, TestFaceting, TestRecovery, TermVectorComponentDistributedTest, DistributedTermsComponentTest, TestCoreContainer, ConvertedLegacyTest, TestFiltering, TestFunctionQuery, TestBadConfig, HighlighterTest, ShowFileRequestHandlerTest, SolrIndexSplitterTest, TestCoreDiscovery, AnalysisAfterCoreReloadTest, SolrRequestParserTest, TestFoldingMultitermQuery, SuggesterTest, SpatialFilterTest, SuggesterWFSTTest, TestCSVLoader, NoCacheHeaderTest, WordBreakSolrSpellCheckerTest, SolrCoreCheckLockOnStartupTest, TestUpdate, FieldMutatingUpdateProcessorTest, SolrInfoBeanTest, XsltUpdateRequestHandlerTest, DebugComponentTest, TestSurroundQueryParser, StandardRequestHandlerTest, TestReversedWildcardFilterFactory, DocumentAnalysisRequestHandlerTest, TestQueryTypes, PrimitiveFieldTypeTest, TermsComponentTest, XmlUpdateRequestHandlerTest, DocumentBuilderTest, PathHierarchyTokenizerFactoryTest, TestIndexingPerformance, MoreLikeThisHandlerTest, TestSolrQueryParser, RegexBoostProcessorTest, IndexSchemaTest, TestJmxIntegration]
[junit4] Completed [406/724 (1!)] on J2 in 0.70s, 3 tests, 1 error, 1 skipped <<< FAILURES!
[...truncated 44390 lines...]
[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1321 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1321/
3 tests failed.
FAILED: org.apache.lucene.analysis.ja.TestFactories.test
Error Message:
Unable to parse
Stack Trace:
java.lang.NumberFormatException: Unable to parse
at __randomizedtesting.SeedInfo.seed([EB650143691E67D2:63313E99C7E20A2A]:0)
at org.apache.lucene.util.ArrayUtil.parse(ArrayUtil.java:94)
at org.apache.lucene.util.ArrayUtil.parseInt(ArrayUtil.java:83)
at org.apache.lucene.util.ArrayUtil.parseInt(ArrayUtil.java:51)
at org.apache.lucene.analysis.miscellaneous.DelimitedTermFrequencyTokenFilter.incrementToken(DelimitedTermFrequencyTokenFilter.java:67)
at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkAnalysisConsistency(BaseTokenStreamTestCase.java:731)
at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:642)
at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:540)
at org.apache.lucene.analysis.ja.TestFactories.doTestTokenFilter(TestFactories.java:102)
at org.apache.lucene.analysis.ja.TestFactories.test(TestFactories.java:55)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.HttpPartitionTest.test
Error Message:
Doc with id=954 not found in http://127.0.0.1:38013/c8n_1x2 due to: Path not found: /id; rsp={doc=null}
Stack Trace:
java.lang.AssertionError: Doc with id=954 not found in http://127.0.0.1:38013/c8n_1x2 due to: Path not found: /id; rsp={doc=null}
at __randomizedtesting.SeedInfo.seed([8DF1F8B5FF38CB9D:5A5C76F51C4A665]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.HttpPartitionTest.assertDocExists(HttpPartitionTest.java:602)
at org.apache.solr.cloud.HttpPartitionTest.assertDocsExistInAllReplicas(HttpPartitionTest.java:557)
at org.apache.solr.cloud.HttpPartitionTest.testRf2(HttpPartitionTest.java:378)
at org.apache.solr.cloud.HttpPartitionTest.test(HttpPartitionTest.java:131)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.addReplicaTest
Error Message:
Error from server at https://127.0.0.1:59685/solr: ADDREPLICA failed to create replica
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:59685/solr: ADDREPLICA failed to create replica
at __randomizedtesting.SeedInfo.seed([8DF1F8B5FF38CB9D:1E51AF937130804C]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:610)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:237)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:459)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:389)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1130)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:806)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:177)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.addReplicaTest(CollectionsAPIDistributedZkTest.java:724)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 4000 lines...]
[junit4] Suite: org.apache.lucene.analysis.ja.TestFactories
[junit4] 2> TEST FAIL: useCharFilter=true text='&sv \ud9ce\udcddS|\u449a\uf28c\uf7ba%\u745e\u05ad \ud7e5 y'
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestFactories -Dtests.method=test -Dtests.seed=EB650143691E67D2 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=uk-UA -Dtests.timezone=Asia/Urumqi -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] ERROR 14.8s J2 | TestFactories.test <<<
[junit4] > Throwable #1: java.lang.NumberFormatException: Unable to parse
[junit4] > at __randomizedtesting.SeedInfo.seed([EB650143691E67D2:63313E99C7E20A2A]:0)
[junit4] > at org.apache.lucene.util.ArrayUtil.parse(ArrayUtil.java:94)
[junit4] > at org.apache.lucene.util.ArrayUtil.parseInt(ArrayUtil.java:83)
[junit4] > at org.apache.lucene.util.ArrayUtil.parseInt(ArrayUtil.java:51)
[junit4] > at org.apache.lucene.analysis.miscellaneous.DelimitedTermFrequencyTokenFilter.incrementToken(DelimitedTermFrequencyTokenFilter.java:67)
[junit4] > at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkAnalysisConsistency(BaseTokenStreamTestCase.java:731)
[junit4] > at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:642)
[junit4] > at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:540)
[junit4] > at org.apache.lucene.analysis.ja.TestFactories.doTestTokenFilter(TestFactories.java:102)
[junit4] > at org.apache.lucene.analysis.ja.TestFactories.test(TestFactories.java:55)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/build/analysis/kuromoji/test/J2/temp/lucene.analysis.ja.TestFactories_EB650143691E67D2-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {dummy=PostingsFormat(name=Direct)}, docValues:{}, maxPointsInLeafNode=693, maxMBSortInHeap=6.805968187498799, sim=RandomSimilarity(queryNorm=false): {dummy=DFR G3(800.0)}, locale=uk-UA, timezone=Asia/Urumqi
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_121 (64-bit)/cpus=4,threads=1,free=219584264,total=292552704
[junit4] 2> NOTE: All tests run in this JVM: [TestJapaneseAnalyzer, TestJapaneseReadingFormFilterFactory, TestToStringUtil, TestTokenInfoDictionary, TestJapaneseIterationMarkCharFilter, TestJapaneseKatakanaStemFilter, TestJapanesePartOfSpeechStopFilterFactory, TestJapaneseBaseFormFilterFactory, TestJapaneseIterationMarkCharFilterFactory, TestJapaneseTokenizerFactory, TestSearchMode, TestJapaneseKatakanaStemFilterFactory, TestFactories]
[junit4] Completed [17/21 (1!)] on J2 in 14.84s, 1 test, 1 error <<< FAILURES!
[...truncated 8414 lines...]
[junit4] Suite: org.apache.solr.cloud.HttpPartitionTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_8DF1F8B5FF38CB9D-001/init-core-data-001
[junit4] 2> 1913584 WARN (SUITE-HttpPartitionTest-seed#[8DF1F8B5FF38CB9D]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
[junit4] 2> 1913585 INFO (SUITE-HttpPartitionTest-seed#[8DF1F8B5FF38CB9D]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields
[junit4] 2> 1913586 INFO (SUITE-HttpPartitionTest-seed#[8DF1F8B5FF38CB9D]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
[junit4] 2> 1913586 INFO (SUITE-HttpPartitionTest-seed#[8DF1F8B5FF38CB9D]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 2> 1913588 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1913589 INFO (Thread-5235) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1913589 INFO (Thread-5235) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1913603 ERROR (Thread-5235) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1913689 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.c.ZkTestServer start zk server on port:33959
[junit4] 2> 1913756 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 1913760 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 1913762 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1913763 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 1913764 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 1913766 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 1913768 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 1913770 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 1913771 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1913773 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 1913775 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 1913777 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
[junit4] 2> 1914024 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_8DF1F8B5FF38CB9D-001/control-001/cores/collection1
[junit4] 2> 1914026 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1914028 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3842a1b8{/,null,AVAILABLE}
[junit4] 2> 1914028 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@79ee1e65{HTTP/1.1,[http/1.1]}{127.0.0.1:45391}
[junit4] 2> 1914028 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.e.j.s.Server Started @1917273ms
[junit4] 2> 1914028 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_8DF1F8B5FF38CB9D-001/tempDir-001/control/data, hostContext=/, hostPort=38013, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_8DF1F8B5FF38CB9D-001/control-001/cores}
[junit4] 2> 1914028 ERROR (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1914029 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.0.0
[junit4] 2> 1914029 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1914029 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1914029 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-06-10T19:06:02.028Z
[junit4] 2> 1914041 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1914042 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_8DF1F8B5FF38CB9D-001/control-001/solr.xml
[junit4] 2> 1914047 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@2242e35, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 1914065 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33959/solr
[junit4] 2> 1914202 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [n:127.0.0.1:38013_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1914204 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [n:127.0.0.1:38013_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:38013_
[junit4] 2> 1914206 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [n:127.0.0.1:38013_ ] o.a.s.c.Overseer Overseer (id=98115358660952068-127.0.0.1:38013_-n_0000000000) starting
[junit4] 2> 1914239 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [n:127.0.0.1:38013_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38013_
[junit4] 2> 1914241 INFO (zkCallback-1814-thread-1-processing-n:127.0.0.1:38013_) [n:127.0.0.1:38013_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1914330 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [n:127.0.0.1:38013_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2242e35
[junit4] 2> 1914336 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [n:127.0.0.1:38013_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2242e35
[junit4] 2> 1914336 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [n:127.0.0.1:38013_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2242e35
[junit4] 2> 1914338 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [n:127.0.0.1:38013_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_8DF1F8B5FF38CB9D-001/control-001/cores
[junit4] 2> 1914338 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [n:127.0.0.1:38013_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1914339 INFO (coreLoadExecutor-4456-thread-1-processing-n:127.0.0.1:38013_) [n:127.0.0.1:38013_ c:control_collection x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 1914345 INFO (OverseerStateUpdate-98115358660952068-127.0.0.1:38013_-n_0000000000) [n:127.0.0.1:38013_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 1915358 INFO (coreLoadExecutor-4456-thread-1-processing-n:127.0.0.1:38013_) [n:127.0.0.1:38013_ c:control_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1915384 INFO (coreLoadExecutor-4456-thread-1-processing-n:127.0.0.1:38013_) [n:127.0.0.1:38013_ c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1915568 INFO (coreLoadExecutor-4456-thread-1-processing-n:127.0.0.1:38013_) [n:127.0.0.1:38013_ c:control_collection x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 1915599 INFO (coreLoadExecutor-4456-thread-1-processing-n:127.0.0.1:38013_) [n:127.0.0.1:38013_ c:control_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection, trusted=true
[junit4] 2> 1915599 INFO (coreLoadExecutor-4456-thread-1-processing-n:127.0.0.1:38013_) [n:127.0.0.1:38013_ c:control_collection x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.core_node1' (registry 'solr.core.control_collection.shard1.core_node1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2242e35
[junit4] 2> 1915600 INFO (coreLoadExecutor-4456-thread-1-processing-n:127.0.0.1:38013_) [n:127.0.0.1:38013_ c:control_collection x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1915600 INFO (coreLoadExecutor-4456-thread-1-processing-n:127.0.0.1:38013_) [n:127.0.0.1:38013_ c:control_collection x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_8DF1F8B5FF38CB9D-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_8DF1F8B5FF38CB9D-001/control-001/cores/collection1/data/]
[junit4] 2> 1915602 INFO (coreLoadExecutor-4456-thread-1-processing-n:127.0.0.1:38013_) [n:127.0.0.1:38013_ c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=240216446, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 1915631 WARN (coreLoadExecutor-4456-thread-1-processing-n:127.0.0.1:38013_) [n:127.0.0.1:38013_ c:control_collection x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1915702 INFO (coreLoadExecutor-4456-thread-1-processing-n:127.0.0.1:38013_) [n:127.0.0.1:38013_ c:control_collection x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1915702 INFO (coreLoadExecutor-4456-thread-1-processing-n:127.0.0.1:38013_) [n:127.0.0.1:38013_ c:control_collection x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1915705 INFO (coreLoadExecutor-4456-thread-1-processing-n:127.0.0.1:38013_) [n:127.0.0.1:38013_ c:control_collection x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1915705 INFO (coreLoadExecutor-4456-thread-1-processing-n:127.0.0.1:38013_) [n:127.0.0.1:38013_ c:control_collection x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1915709 INFO (coreLoadExecutor-4456-thread-1-processing-n:127.0.0.1:38013_) [n:127.0.0.1:38013_ c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=39, maxMergeAtOnceExplicit=38, maxMergedSegmentMB=6.263671875, floorSegmentMB=0.470703125, forceMergeDeletesPctAllowed=17.41736794585829, segmentsPerTier=37.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 1915709 INFO (coreLoadExecutor-4456-thread-1-processing-n:127.0.0.1:38013_) [n:127.0.0.1:38013_ c:control_collection x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5bdeb384[collection1] main]
[junit4] 2> 1915723 INFO (coreLoadExecutor-4456-thread-1-processing-n:127.0.0.1:38013_) [n:127.0.0.1:38013_ c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1915728 INFO (coreLoadExecutor-4456-thread-1-processing-n:127.0.0.1:38013_) [n:127.0.0.1:38013_ c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1915729 INFO (coreLoadExecutor-4456-thread-1-processing-n:127.0.0.1:38013_) [n:127.0.0.1:38013_ c:control_collection x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1915729 INFO (coreLoadExecutor-4456-thread-1-processing-n:127.0.0.1:38013_) [n:127.0.0.1:38013_ c:control_collection x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1569845740807651328
[junit4] 2> 1915733 INFO (searcherExecutor-4457-thread-1-processing-n:127.0.0.1:38013_ x:collection1 c:control_collection) [n:127.0.0.1:38013_ c:control_collection x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5bdeb384[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1915779 INFO (coreZkRegister-4449-thread-1-processing-n:127.0.0.1:38013_ x:collection1 c:control_collection) [n:127.0.0.1:38013_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1915779 INFO (coreZkRegister-4449-thread-1-processing-n:127.0.0.1:38013_ x:collection1 c:control_collection) [n:127.0.0.1:38013_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1915779 INFO (coreZkRegister-4449-thread-1-processing-n:127.0.0.1:38013_ x:collection1 c:control_collection) [n:127.0.0.1:38013_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:38013/collection1/
[junit4] 2> 1915779 INFO (coreZkRegister-4449-thread-1-processing-n:127.0.0.1:38013_ x:collection1 c:control_collection) [n:127.0.0.1:38013_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1915779 INFO (coreZkRegister-4449-thread-1-processing-n:127.0.0.1:38013_ x:collection1 c:control_collection) [n:127.0.0.1:38013_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:38013/collection1/ has no replicas
[junit4] 2> 1915779 INFO (coreZkRegister-4449-thread-1-processing-n:127.0.0.1:38013_ x:collection1 c:control_collection) [n:127.0.0.1:38013_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 1915802 INFO (coreZkRegister-4449-thread-1-processing-n:127.0.0.1:38013_ x:collection1 c:control_collection) [n:127.0.0.1:38013_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:38013/collection1/ shard1
[junit4] 2> 1915872 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1915873 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33959/solr ready
[junit4] 2> 1915873 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 1915956 INFO (coreZkRegister-4449-thread-1-processing-n:127.0.0.1:38013_ x:collection1 c:control_collection) [n:127.0.0.1:38013_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1916077 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_8DF1F8B5FF38CB9D-001/shard-1-001/cores/collection1
[junit4] 2> 1916078 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_8DF1F8B5FF38CB9D-001/shard-1-001 of type NRT
[junit4] 2> 1916079 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1916082 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@28dca412{/,null,AVAILABLE}
[junit4] 2> 1916082 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@668f210a{HTTP/1.1,[http/1.1]}{127.0.0.1:35660}
[junit4] 2> 1916082 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.e.j.s.Server Started @1919327ms
[junit4] 2> 1916082 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_8DF1F8B5FF38CB9D-001/tempDir-001/jetty1, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/, hostPort=51705, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_8DF1F8B5FF38CB9D-001/shard-1-001/cores}
[junit4] 2> 1916083 ERROR (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1916091 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.0.0
[junit4] 2> 1916091 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1916091 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1916092 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-06-10T19:06:04.091Z
[junit4] 2> 1916138 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1916138 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_8DF1F8B5FF38CB9D-001/shard-1-001/solr.xml
[junit4] 2> 1916143 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@2242e35, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 1916161 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33959/solr
[junit4] 2> 1916265 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [n:127.0.0.1:51705_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1916267 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [n:127.0.0.1:51705_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1916270 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [n:127.0.0.1:51705_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51705_
[junit4] 2> 1916271 INFO (zkCallback-1814-thread-1-processing-n:127.0.0.1:38013_) [n:127.0.0.1:38013_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1916271 INFO (zkCallback-1818-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1916289 INFO (zkCallback-1823-thread-2-processing-n:127.0.0.1:51705_) [n:127.0.0.1:51705_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1916389 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [n:127.0.0.1:51705_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2242e35
[junit4] 2> 1916396 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [n:127.0.0.1:51705_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2242e35
[junit4] 2> 1916396 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [n:127.0.0.1:51705_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2242e35
[junit4] 2> 1916398 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [n:127.0.0.1:51705_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_8DF1F8B5FF38CB9D-001/shard-1-001/cores
[junit4] 2> 1916398 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [n:127.0.0.1:51705_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1916399 INFO (coreLoadExecutor-4467-thread-1-processing-n:127.0.0.1:51705_) [n:127.0.0.1:51705_ c:collection1 x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 1916401 INFO (OverseerStateUpdate-98115358660952068-127.0.0.1:38013_-n_0000000000) [n:127.0.0.1:38013_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 1917420 INFO (coreLoadExecutor-4467-thread-1-processing-n:127.0.0.1:51705_) [n:127.0.0.1:51705_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1917464 INFO (coreLoadExecutor-4467-thread-1-processing-n:127.0.0.1:51705_) [n:127.0.0.1:51705_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1917574 INFO (coreLoadExecutor-4467-thread-1-processing-n:127.0.0.1:51705_) [n:127.0.0.1:51705_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 1917587 INFO (coreLoadExecutor-4467-thread-1-processing-n:127.0.0.1:51705_) [n:127.0.0.1:51705_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1, trusted=true
[junit4] 2> 1917587 INFO (coreLoadExecutor-4467-thread-1-processing-n:127.0.0.1:51705_) [n:127.0.0.1:51705_ c:collection1 x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.core_node1' (registry 'solr.core.collection1.shard2.core_node1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2242e35
[junit4] 2> 1917587 INFO (coreLoadExecutor-4467-thread-1-processing-n:127.0.0.1:51705_) [n:127.0.0.1:51705_ c:collection1 x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1917587 INFO (coreLoadExecutor-4467-thread-1-processing-n:127.0.0.1:51705_) [n:127.0.0.1:51705_ c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_8DF1F8B5FF38CB9D-001/shard-1-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_8DF1F8B5FF38CB9D-001/shard-1-001/cores/collection1/data/]
[junit4] 2> 1917600 INFO (coreLoadExecutor-4467-thread-1-processing-n:127.0.0.1:51705_) [n:127.0.0.1:51705_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=240216446, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 1917603 WARN (coreLoadExecutor-4467-thread-1-processing-n:127.0.0.1:51705_) [n:127.0.0.1:51705_ c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1917672 INFO (coreLoadExecutor-4467-thread-1-processing-n:127.0.0.1:51705_) [n:127.0.0.1:51705_ c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1917672 INFO (coreLoadExecutor-4467-thread-1-processing-n:127.0.0.1:51705_) [n:127.0.0.1:51705_ c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1917673 INFO (coreLoadExecutor-4467-thread-1-processing-n:127.0.0.1:51705_) [n:127.0.0.1:51705_ c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1917673 INFO (coreLoadExecutor-4467-thread-1-processing-n:127.0.0.1:51705_) [n:127.0.0.1:51705_ c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1917675 INFO (coreLoadExecutor-4467-thread-1-processing-n:127.0.0.1:51705_) [n:127.0.0.1:51705_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=39, maxMergeAtOnceExplicit=38, maxMergedSegmentMB=6.263671875, floorSegmentMB=0.470703125, forceMergeDeletesPctAllowed=17.41736794585829, segmentsPerTier=37.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 1917675 INFO (coreLoadExecutor-4467-thread-1-processing-n:127.0.0.1:51705_) [n:127.0.0.1:51705_ c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1d4f0912[collection1] main]
[junit4] 2> 1917676 INFO (coreLoadExecutor-4467-thread-1-processing-n:127.0.0.1:51705_) [n:127.0.0.1:51705_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1917677 INFO (coreLoadExecutor-4467-thread-1-processing-n:127.0.0.1:51705_) [n:127.0.0.1:51705_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1917677 INFO (coreLoadExecutor-4467-thread-1-processing-n:127.0.0.1:51705_) [n:127.0.0.1:51705_ c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1917677 INFO (coreLoadExecutor-4467-thread-1-processing-n:127.0.0.1:51705_) [n:127.0.0.1:51705_ c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1569845742850277376
[junit4] 2> 1917683 INFO (searcherExecutor-4468-thread-1-processing-n:127.0.0.1:51705_ x:collection1 c:collection1) [n:127.0.0.1:51705_ c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1d4f0912[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1917688 INFO (coreZkRegister-4462-thread-1-processing-n:127.0.0.1:51705_ x:collection1 c:collection1) [n:127.0.0.1:51705_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1917688 INFO (coreZkRegister-4462-thread-1-processing-n:127.0.0.1:51705_ x:collection1 c:collection1) [n:127.0.0.1:51705_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1917688 INFO (coreZkRegister-4462-thread-1-processing-n:127.0.0.1:51705_ x:collection1 c:collection1) [n:127.0.0.1:51705_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:51705/collection1/
[junit4] 2> 1917688 INFO (coreZkRegister-4462-thread-1-processing-n:127.0.0.1:51705_ x:collection1 c:collection1) [n:127.0.0.1:51705_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1917688 INFO (coreZkRegister-4462-thread-1-processing-n:127.0.0.1:51705_ x:collection1 c:collection1) [n:127.0.0.1:51705_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:51705/collection1/ has no replicas
[junit4] 2> 1917688 INFO (coreZkRegister-4462-thread-1-processing-n:127.0.0.1:51705_ x:collection1 c:collection1) [n:127.0.0.1:51705_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 1917693 INFO (coreZkRegister-4462-thread-1-processing-n:127.0.0.1:51705_ x:collection1 c:collection1) [n:127.0.0.1:51705_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:51705/collection1/ shard2
[junit4] 2> 1917844 INFO (coreZkRegister-4462-thread-1-processing-n:127.0.0.1:51705_ x:collection1 c:collection1) [n:127.0.0.1:51705_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1918083 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_8DF1F8B5FF38CB9D-001/shard-2-001/cores/collection1
[junit4] 2> 1918084 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_8DF1F8B5FF38CB9D-001/shard-2-001 of type NRT
[junit4] 2> 1918088 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1918089 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4b231ff1{/,null,AVAILABLE}
[junit4] 2> 1918090 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@49150cf3{HTTP/1.1,[http/1.1]}{127.0.0.1:49759}
[junit4] 2> 1918090 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.e.j.s.Server Started @1921335ms
[junit4] 2> 1918090 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_8DF1F8B5FF38CB9D-001/tempDir-001/jetty2, solrconfig=solrconfig.xml, hostContext=/, hostPort=56123, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_8DF1F8B5FF38CB9D-001/shard-2-001/cores}
[junit4] 2> 1918090 ERROR (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1918097 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.0.0
[junit4] 2> 1918097 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1918097 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1918097 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-06-10T19:06:06.096Z
[junit4] 2> 1918160 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1918160 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_8DF1F8B5FF38CB9D-001/shard-2-001/solr.xml
[junit4] 2> 1918166 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@2242e35, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 1918170 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33959/solr
[junit4] 2> 1918198 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [n:127.0.0.1:56123_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1918201 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [n:127.0.0.1:56123_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1918206 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [n:127.0.0.1:56123_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56123_
[junit4] 2> 1918207 INFO (zkCallback-1823-thread-2-processing-n:127.0.0.1:51705_) [n:127.0.0.1:51705_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1918208 INFO (zkCallback-1818-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1918209 INFO (zkCallback-1814-thread-1-processing-n:127.0.0.1:38013_) [n:127.0.0.1:38013_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1918216 INFO (zkCallback-1829-thread-1-processing-n:127.0.0.1:56123_) [n:127.0.0.1:56123_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1918320 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [n:127.0.0.1:56123_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2242e35
[junit4] 2> 1918341 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [n:127.0.0.1:56123_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2242e35
[junit4] 2> 1918342 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [n:127.0.0.1:56123_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2242e35
[junit4] 2> 1918344 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [n:127.0.0.1:56123_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_8DF1F8B5FF38CB9D-001/shard-2-001/cores
[junit4] 2> 1918344 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [n:127.0.0.1:56123_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1918344 INFO (coreLoadExecutor-4478-thread-1-processing-n:127.0.0.1:56123_) [n:127.0.0.1:56123_ c:collection1 x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 1918346 INFO (OverseerStateUpdate-98115358660952068-127.0.0.1:38013_-n_0000000000) [n:127.0.0.1:38013_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 1919376 INFO (coreLoadExecutor-4478-thread-1-processing-n:127.0.0.1:56123_) [n:127.0.0.1:56123_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1919390 INFO (coreLoadExecutor-4478-thread-1-processing-n:127.0.0.1:56123_) [n:127.0.0.1:56123_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1919544 INFO (coreLoadExecutor-4478-thread-1-processing-n:127.0.0.1:56123_) [n:127.0.0.1:56123_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 1919569 INFO (coreLoadExecutor-4478-thread-1-processing-n:127.0.0.1:56123_) [n:127.0.0.1:56123_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1, trusted=true
[junit4] 2> 1919570 INFO (coreLoadExecutor-4478-thread-1-processing-n:127.0.0.1:56123_) [n:127.0.0.1:56123_ c:collection1 x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.core_node2' (registry 'solr.core.collection1.shard1.core_node2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2242e35
[junit4] 2> 1919571 INFO (coreLoadExecutor-4478-thread-1-processing-n:127.0.0.1:56123_) [n:127.0.0.1:56123_ c:collection1 x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1919571 INFO (coreLoadExecutor-4478-thread-1-processing-n:127.0.0.1:56123_) [n:127.0.0.1:56123_ c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_8DF1F8B5FF38CB9D-001/shard-2-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_8DF1F8B5FF38CB9D-001/shard-2-001/cores/collection1/data/]
[junit4] 2> 1919577 INFO (coreLoadExecutor-4478-thread-1-processing-n:127.0.0.1:56123_) [n:127.0.0.1:56123_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=240216446, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 1919580 WARN (coreLoadExecutor-4478-thread-1-processing-n:127.0.0.1:56123_) [n:127.0.0.1:56123_ c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1919647 INFO (coreLoadExecutor-4478-thread-1-processing-n:127.0.0.1:56123_) [n:127.0.0.1:56123_ c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1919647 INFO (coreLoadExecutor-4478-thread-1-processing-n:127.0.0.1:56123_) [n:127.0.0.1:56123_ c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1919649 INFO (coreLoadExecutor-4478-thread-1-processing-n:127.0.0.1:56123_) [n:127.0.0.1:56123_ c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1919649 INFO (coreLoadExecutor-4478-thread-1-processing-n:127.0.0.1:56123_) [n:127.0.0.1:56123_ c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1919650 INFO (coreLoadExecutor-4478-thread-1-processing-n:127.0.0.1:56123_) [n:127.0.0.1:56123_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=39, maxMergeAtOnceExplicit=38, maxMergedSegmentMB=6.263671875, floorSegmentMB=0.470703125, forceMergeDeletesPctAllowed=17.41736794585829, segmentsPerTier=37.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 1919650 INFO (coreLoadExecutor-4478-thread-1-processing-n:127.0.0.1:56123_) [n:127.0.0.1:56123_ c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@241d4f28[collection1] main]
[junit4] 2> 1919652 INFO (coreLoadExecutor-4478-thread-1-processing-n:127.0.0.1:56123_) [n:127.0.0.1:56123_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1919653 INFO (coreLoadExecutor-4478-thread-1-processing-n:127.0.0.1:56123_) [n:127.0.0.1:56123_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1919653 INFO (coreLoadExecutor-4478-thread-1-processing-n:127.0.0.1:56123_) [n:127.0.0.1:56123_ c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1919653 INFO (coreLoadExecutor-4478-thread-1-processing-n:127.0.0.1:56123_) [n:127.0.0.1:56123_ c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1569845744922263552
[junit4] 2> 1919660 INFO (searcherExecutor-4479-thread-1-processing-n:127.0.0.1:56123_ x:collection1 c:collection1) [n:127.0.0.1:56123_ c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@241d4f28[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1919672 INFO (coreZkRegister-4473-thread-1-processing-n:127.0.0.1:56123_ x:collection1 c:collection1) [n:127.0.0.1:56123_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1919672 INFO (coreZkRegister-4473-thread-1-processing-n:127.0.0.1:56123_ x:collection1 c:collection1) [n:127.0.0.1:56123_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1919672 INFO (coreZkRegister-4473-thread-1-processing-n:127.0.0.1:56123_ x:collection1 c:collection1) [n:127.0.0.1:56123_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:56123/collection1/
[junit4] 2> 1919672 INFO (coreZkRegister-4473-thread-1-processing-n:127.0.0.1:56123_ x:collection1 c:collection1) [n:127.0.0.1:56123_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1919672 INFO (coreZkRegister-4473-thread-1-processing-n:127.0.0.1:56123_ x:collection1 c:collection1) [n:127.0.0.1:56123_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:56123/collection1/ has no replicas
[junit4] 2> 1919672 INFO (coreZkRegister-4473-thread-1-processing-n:127.0.0.1:56123_ x:collection1 c:collection1) [n:127.0.0.1:56123_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 1919683 INFO (coreZkRegister-4473-thread-1-processing-n:127.0.0.1:56123_ x:collection1 c:collection1) [n:127.0.0.1:56123_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:56123/collection1/ shard1
[junit4] 2> 1919834 INFO (coreZkRegister-4473-thread-1-processing-n:127.0.0.1:56123_ x:collection1 c:collection1) [n:127.0.0.1:56123_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1920011 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_8DF1F8B5FF38CB9D-001/shard-3-001/cores/collection1
[junit4] 2> 1920012 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_8DF1F8B5FF38CB9D-001/shard-3-001 of type NRT
[junit4] 2> 1920013 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1920024 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1ca640b2{/,null,AVAILABLE}
[junit4] 2> 1920025 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@21a83e12{HTTP/1.1,[http/1.1]}{127.0.0.1:41606}
[junit4] 2> 1920025 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.e.j.s.Server Started @1923270ms
[junit4] 2> 1920025 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_8DF1F8B5FF38CB9D-001/tempDir-001/jetty3, solrconfig=solrconfig.xml, hostContext=/, hostPort=49572, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_8DF1F8B5FF38CB9D-001/shard-3-001/cores}
[junit4] 2> 1920025 ERROR (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1920026 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.0.0
[junit4] 2> 1920026 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1920026 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1920026 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-06-10T19:06:08.025Z
[junit4] 2> 1920049 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1920049 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_8DF1F8B5FF38CB9D-001/shard-3-001/solr.xml
[junit4] 2> 1920055 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@2242e35, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 1920074 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33959/solr
[junit4] 2> 1920094 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [n:127.0.0.1:49572_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 1920096 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [n:127.0.0.1:49572_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1920099 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [n:127.0.0.1:49572_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49572_
[junit4] 2> 1920100 INFO (zkCallback-1823-thread-2-processing-n:127.0.0.1:51705_) [n:127.0.0.1:51705_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1920100 INFO (zkCallback-1818-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1920101 INFO (zkCallback-1829-thread-1-processing-n:127.0.0.1:56123_) [n:127.0.0.1:56123_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1920102 INFO (zkCallback-1814-thread-1-processing-n:127.0.0.1:38013_) [n:127.0.0.1:38013_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1920112 INFO (zkCallback-1835-thread-1-processing-n:127.0.0.1:49572_) [n:127.0.0.1:49572_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1920211 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [n:127.0.0.1:49572_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2242e35
[junit4] 2> 1920217 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [n:127.0.0.1:49572_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2242e35
[junit4] 2> 1920217 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [n:127.0.0.1:49572_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2242e35
[junit4] 2> 1920220 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [n:127.0.0.1:49572_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_8DF1F8B5FF38CB9D-001/shard-3-001/cores
[junit4] 2> 1920220 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [n:127.0.0.1:49572_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1920220 INFO (coreLoadExecutor-4489-thread-1-processing-n:127.0.0.1:49572_) [n:127.0.0.1:49572_ c:collection1 x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 1920222 INFO (OverseerStateUpdate-98115358660952068-127.0.0.1:38013_-n_0000000000) [n:127.0.0.1:38013_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 1921246 INFO (coreLoadExecutor-4489-thread-1-processing-n:127.0.0.1:49572_) [n:127.0.0.1:49572_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1921261 INFO (coreLoadExecutor-4489-thread-1-processing-n:127.0.0.1:49572_) [n:127.0.0.1:49572_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1921401 INFO (coreLoadExecutor-4489-thread-1-processing-n:127.0.0.1:49572_) [n:127.0.0.1:49572_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 1921420 INFO (coreLoadExecutor-4489-thread-1-processing-n:127.0.0.1:49572_) [n:127.0.0.1:49572_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1, trusted=true
[junit4] 2> 1921421 INFO (coreLoadExecutor-4489-thread-1-processing-n:127.0.0.1:49572_) [n:127.0.0.1:49572_ c:collection1 x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.core_node3' (registry 'solr.core.collection1.shard2.core_node3') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2242e35
[junit4] 2> 1921421 INFO (coreLoadExecutor-4489-thread-1-processing-n:127.0.0.1:49572_) [n:127.0.0.1:49572_ c:collection1 x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1921421 INFO (coreLoadExecutor-4489-thread-1-processing-n:127.0.0.1:49572_) [n:127.0.0.1:49572_ c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_8DF1F8B5FF38CB9D-001/shard-3-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_8DF1F8B5FF38CB9D-001/shard-3-001/cores/collection1/data/]
[junit4] 2> 1921423 INFO (coreLoadExecutor-4489-thread-1-processing-n:127.0.0.1:49572_) [n:127.0.0.1:49572_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=240216446, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 1921426 WARN (coreLoadExecutor-4489-thread-1-processing-n:127.0.0.1:49572_) [n:127.0.0.1:49572_ c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1921490 INFO (coreLoadExecutor-4489-thread-1-processing-n:127.0.0.1:49572_) [n:127.0.0.1:49572_ c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1921490 INFO (coreLoadExecutor-4489-thread-1-processing-n:127.0.0.1:49572_) [n:127.0.0.1:49572_ c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1921491 INFO (coreLoadExecutor-4489-thread-1-processing-n:127.0.0.1:49572_) [n:127.0.0.1:49572_ c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1921491 INFO (coreLoadExecutor-4489-thread-1-processing-n:127.0.0.1:49572_) [n:127.0.0.1:49572_ c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1921495 INFO (coreLoadExecutor-4489-thread-1-processing-n:127.0.0.1:49572_) [n:127.0.0.1:49572_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=39, maxMergeAtOnceExplicit=38, maxMergedSegmentMB=6.263671875, floorSegmentMB=0.470703125, forceMergeDeletesPctAllowed=17.41736794585829, segmentsPerTier=37.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 1921496 INFO (coreLoadExecutor-4489-thread-1-processing-n:127.0.0.1:49572_) [n:127.0.0.1:49572_ c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@239fdbff[collection1] main]
[junit4] 2> 1921497 INFO (coreLoadExecutor-4489-thread-1-processing-n:127.0.0.1:49572_) [n:127.0.0.1:49572_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1921498 INFO (coreLoadExecutor-4489-thread-1-processing-n:127.0.0.1:49572_) [n:127.0.0.1:49572_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1921498 INFO (coreLoadExecutor-4489-thread-1-processing-n:127.0.0.1:49572_) [n:127.0.0.1:49572_ c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1921500 INFO (searcherExecutor-4490-thread-1-processing-n:127.0.0.1:49572_ x:collection1 c:collection1) [n:127.0.0.1:49572_ c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@239fdbff[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1921501 INFO (coreLoadExecutor-4489-thread-1-processing-n:127.0.0.1:49572_) [n:127.0.0.1:49572_ c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1569845746860032000
[junit4] 2> 1921510 INFO (coreZkRegister-4484-thread-1-processing-n:127.0.0.1:49572_ x:collection1 c:collection1) [n:127.0.0.1:49572_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.ZkController Core needs to recover:collection1
[junit4] 2> 1921511 INFO (updateExecutor-1832-thread-1-processing-n:127.0.0.1:49572_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:49572_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.DefaultSolrCoreState Running recovery
[junit4] 2> 1921511 INFO (recoveryExecutor-1833-thread-1-processing-n:127.0.0.1:49572_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:49572_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
[junit4] 2> 1921512 INFO (recoveryExecutor-1833-thread-1-processing-n:127.0.0.1:49572_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:49572_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
[junit4] 2> 1921512 INFO (recoveryExecutor-1833-thread-1-processing-n:127.0.0.1:49572_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:49572_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1]
[junit4] 2> 1921512 INFO (recoveryExecutor-1833-thread-1-processing-n:127.0.0.1:49572_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:49572_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 1921512 INFO (recoveryExecutor-1833-thread-1-processing-n:127.0.0.1:49572_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:49572_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Publishing state of core [collection1] as recovering, leader is [http://127.0.0.1:51705/collection1/] and I am [http://127.0.0.1:49572/collection1/]
[junit4] 2> 1921517 INFO (recoveryExecutor-1833-thread-1-processing-n:127.0.0.1:49572_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:49572_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:51705]; [WaitForState: action=PREPRECOVERY&core=collection1&nodeName=127.0.0.1:49572_&coreNodeName=core_node3&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
[junit4] 2> 1921517 INFO (SocketProxy-Acceptor-51705) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=45871,localport=51705], receiveBufferSize:531000
[junit4] 2> 1921523 INFO (SocketProxy-Acceptor-51705) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=35660,localport=57868], receiveBufferSize=530904
[junit4] 2> 1921525 INFO (qtp800971373-13924) [n:127.0.0.1:51705_ ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true, maxTime: 183 s
[junit4] 2> 1921527 INFO (qtp800971373-13924) [n:127.0.0.1:51705_ ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard2, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:49572_, coreNodeName=core_node3, onlyIfActiveCheckResult=false, nodeProps: core_node3:{"core":"collection1","base_url":"http://127.0.0.1:49572","node_name":"127.0.0.1:49572_","state":"down","type":"NRT"}
[junit4] 2> 1921811 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.SolrTestCaseJ4 ###Starting test
[junit4] 2> 1921811 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Wait for recoveries to finish - wait 30000 for each attempt
[junit4] 2> 1921811 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):30000
[junit4] 2> 1922527 INFO (qtp800971373-13924) [n:127.0.0.1:51705_ ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard2, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:49572_, coreNodeName=core_node3, onlyIfActiveCheckResult=false, nodeProps: core_node3:{"core":"collection1","base_url":"http://127.0.0.1:49572","node_name":"127.0.0.1:49572_","state":"recovering","type":"NRT"}
[junit4] 2> 1922527 INFO (qtp800971373-13924) [n:127.0.0.1:51705_ ] o.a.s.h.a.PrepRecoveryOp Waited coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4] 2> 1922527 INFO (qtp800971373-13924) [n:127.0.0.1:51705_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:49572_&onlyIfLeaderActive=true&core=collection1&coreNodeName=core_node3&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=1002
[junit4] 2> 1923028 INFO (recoveryExecutor-1833-thread-1-processing-n:127.0.0.1:49572_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:49572_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Attempting to PeerSync from [http://127.0.0.1:51705/collection1/] - recoveringAfterStartup=[true]
[junit4] 2> 1923028 INFO (recoveryExecutor-1833-thread-1-processing-n:127.0.0.1:49572_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:49572_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.PeerSync PeerSync: core=collection1 url=http://127.0.0.1:49572 START replicas=[http://127.0.0.1:51705/collection1/] nUpdates=100
[junit4] 2> 1923029 INFO (SocketProxy-Acceptor-51705) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=45874,localport=51705], receiveBufferSize:531000
[junit4] 2> 1923034 INFO (SocketProxy-Acceptor-51705) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=35660,localport=57871], receiveBufferSize=530904
[junit4] 2> 1923046 INFO (qtp800971373-13928) [n:127.0.0.1:51705_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.IndexFingerprint IndexFingerprint millis:4.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
[junit4] 2> 1923046 INFO (qtp800971373-13928) [n:127.0.0.1:51705_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/get params={distrib=false&qt=/get&getFingerprint=9223372036854775807&wt=javabin&version=2} status=0 QTime=4
[junit4] 2> 1923047 INFO (recoveryExecutor-1833-thread-1-processing-n:127.0.0.1:49572_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:49572_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.IndexFingerprint IndexFingerprint millis:0.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
[junit4] 2> 1923047 INFO (recoveryExecutor-1833-thread-1-processing-n:127.0.0.1:49572_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:49572_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.PeerSync We are already in sync. No need to do a PeerSync
[junit4] 2> 1923047 INFO (recoveryExecutor-1833-thread-1-processing-n:127.0.0.1:49572_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:49572_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 1923047 INFO (recoveryExecutor-1833-thread-1-processing-n:127.0.0.1:49572_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:49572_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 1923048 INFO (recoveryExecutor-1833-thread-1-processing-n:127.0.0.1:49572_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:49572_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 1923048 INFO (recoveryExecutor-1833-thread-1-processing-n:127.0.0.1:49572_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:49572_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy PeerSync stage of recovery was successful.
[junit4] 2> 1923048 INFO (recoveryExecutor-1833-thread-1-processing-n:127.0.0.1:49572_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:49572_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Replaying updates buffered during PeerSync.
[junit4] 2> 1923048 INFO (recoveryExecutor-1833-thread-1-processing-n:127.0.0.1:49572_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:49572_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy No replay needed.
[junit4] 2> 1923048 INFO (recoveryExecutor-1833-thread-1-processing-n:127.0.0.1:49572_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:49572_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Registering as Active after recovery.
[junit4] 2> 1923811 INFO (TEST-HttpPartitionTest.test-seed#[8DF1F8B5FF38CB9D]) [ ] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: collection1
[junit4] 2> 1923812 INFO (SocketProxy-Acceptor-38013) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=55547,localport=38013], receiveBufferSize:531000
[junit4] 2> 1923815 INFO (SocketProxy-Acceptor-38013) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=45391,localport=38131], receiveBufferSize=530904
[junit4] 2> 1923815 INFO (qtp2064796709-13888) [n:127.0.0.1:38013_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1569845749286436864,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 1923816 INFO (qtp2064796709-13888) [n:127.0.0.1:38013_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 1923817 INFO (qtp2064796709-13888) [n:127.0.0.1:38013_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 1923817 INFO (qtp2064796709-13888) [n:127.0.0.1:38013_ c:contro
[...truncated too long message...]
(MetricsSourceAdapter.java:211)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:463)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:213)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:594)
[junit4] 2> at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
[junit4] 2> at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
[junit4] 2> at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:171)
[junit4] 2> at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:872)
[junit4] 2> at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1726)
[junit4] 2> at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1705)
[junit4] 2> at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:204)
[junit4] 2> at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.teardownClass(HdfsCollectionsAPIDistributedZkTest.java:77)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> Caused by: java.lang.NullPointerException
[junit4] 2> at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:203)
[junit4] 2> at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3370)
[junit4] 2> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:5729)
[junit4] 2> ... 54 more
[junit4] 2> 3390337 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[8DF1F8B5FF38CB9D]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:58434 58434
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_8DF1F8B5FF38CB9D-001
[junit4] 2> Jun 10, 2017 7:30:38 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 18 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {_version_=PostingsFormat(name=Direct), id=Lucene50(blocksize=128)}, docValues:{}, maxPointsInLeafNode=1420, maxMBSortInHeap=5.226768009285871, sim=RandomSimilarity(queryNorm=true): {}, locale=sr-Latn, timezone=America/Argentina/Cordoba
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_121 (64-bit)/cpus=4,threads=2,free=393453856,total=509083648
[junit4] 2> NOTE: All tests run in this JVM: [ChangedSchemaMergeTest, AutoCommitTest, CleanupOldIndexTest, TestLegacyNumericUtils, ForceLeaderTest, TestManagedSchemaThreadSafety, FullSolrCloudDistribCmdsTest, TestLockTree, SolrCloudExampleTest, SolrTestCaseJ4Test, CurrencyFieldXmlFileTest, DistributedFacetExistsSmallTest, SimpleFacetsTest, BaseCdcrDistributedZkTest, NoCacheHeaderTest, CachingDirectoryFactoryTest, TestSmileRequest, StatelessScriptUpdateProcessorFactoryTest, ExternalFileFieldSortTest, DirectoryFactoryTest, TestBlendedInfixSuggestions, RequestHandlersTest, TestGraphMLResponseWriter, TestFieldCollectionResource, TestLMDirichletSimilarityFactory, TestChildDocTransformer, TestHighFrequencyDictionaryFactory, TestDocTermOrds, TestQuerySenderNoQuery, TestConfigSetImmutable, FacetPivotSmallTest, TestUtils, TestSchemaSimilarityResource, TestLuceneMatchVersion, JSONWriterTest, HLLUtilTest, GraphQueryTest, TestSolrCloudWithKerberosAlt, TestDFISimilarityFactory, OverseerModifyCollectionTest, TestCoreDiscovery, BasicAuthIntegrationTest, TestFieldTypeResource, CheckHdfsIndexTest, DeleteInactiveReplicaTest, CollectionStateFormat2Test, TestComponentsName, LukeRequestHandlerTest, BasicDistributedZk2Test, ParsingFieldUpdateProcessorsTest, TestCollationField, TestExclusionRuleCollectionAccess, EnumFieldTest, TestNumericTerms64, HdfsChaosMonkeyNothingIsSafeTest, ChaosMonkeyNothingIsSafeWithPullReplicasTest, SolrShardReporterTest, BadIndexSchemaTest, DistribCursorPagingTest, LeaderFailoverAfterPartitionTest, BigEndianAscendingWordDeserializerTest, TestOrdValues, TestNRTOpen, ExplicitHLLTest, TestClusterProperties, DateFieldTest, CircularListTest, CloudMLTQParserTest, SolrJmxReporterTest, TestConfig, TestSort, TestTestInjection, TestShortCircuitedRequests, OpenExchangeRatesOrgProviderTest, StatsComponentTest, ShowFileRequestHandlerTest, TestSimpleTextCodec, UnloadDistributedZkTest, HdfsTlogReplayBufferedWhileIndexingTest, ChaosMonkeyNothingIsSafeTest, TestObjectReleaseTracker, TestHttpShardHandlerFactory, TestPayloadScoreQParserPlugin, DebugComponentTest, TestUniqueKeyFieldResource, DistributedQueueTest, TestSolrCloudWithHadoopAuthPlugin, TestPullReplicaErrorHandling, TriLevelCompositeIdRoutingTest, OverseerTaskQueueTest, XmlUpdateRequestHandlerTest, TestStressInPlaceUpdates, TestWriterPerf, PrimitiveFieldTypeTest, CoreAdminHandlerTest, TestRestManager, TestFieldSortValues, TestCloudPivotFacet, SharedFSAutoReplicaFailoverUtilsTest, DocValuesMultiTest, DistributedQueryComponentCustomSortTest, SpatialRPTFieldTypeTest, QueryResultKeyTest, CoreAdminCreateDiscoverTest, TestIntervalFaceting, TermVectorComponentDistributedTest, SolrMetricsIntegrationTest, TestFieldCacheSort, TestMiniSolrCloudClusterSSL, SuggestComponentTest, TemplateUpdateProcessorTest, TestMergePolicyConfig, TestManagedSynonymGraphFilterFactory, TestHashQParserPlugin, FieldMutatingUpdateProcessorTest, CollectionsAPIDistributedZkTest, SyncSliceTest, OpenCloseCoreStressTest, LeaderElectionTest, TestZkChroot, ZkCLITest, TestDistributedSearch, TestDistributedGrouping, TestRecovery, TestHashPartitioner, TestRealTimeGet, TestStressReorder, TestReload, HardAutoCommitTest, TestSolr4Spatial, SolrCmdDistributorTest, ConvertedLegacyTest, TestFiltering, SoftAutoCommitTest, TestFuzzyAnalyzedSuggestions, TestPhraseSuggestions, SpellPossibilityIteratorTest, SynonymTokenizerTest, TestXIncludeConfig, TestPerFieldSimilarity, TestBM25SimilarityFactory, ResourceLoaderTest, ScriptEngineTest, PreAnalyzedFieldTest, DOMUtilTest, ClusterStateTest, TestDocumentBuilder, FileUtilsTest, TestCrossCoreJoin, TestCursorMarkWithoutUniqueKey, TestDistributedMissingSort, TestHighlightDedupGrouping, TestSimpleTrackingShardHandler, TestTolerantSearch, TestEmbeddedSolrServerConstructors, TestEmbeddedSolrServerSchemaAPI, TestJettySolrRunner, ConnectionReuseTest, ActionThrottleTest, AliasIntegrationTest, CdcrRequestHandlerTest, CollectionsAPISolrJTest, DistribDocExpirationUpdateProcessorTest, MultiThreadedOCPTest, SimpleCollectionCreateDeleteTest, SolrXmlInZkTest, TestCloudDeleteByQuery, TestCloudInspectUtil, TestCloudPseudoReturnFields, TestCloudRecovery, TestCollectionAPI, TestConfigSetsAPIExclusivity, TestConfigSetsAPIZkFailure, TestDeleteCollectionOnDownNodes, TestDownShardTolerantSearch, TestLeaderElectionWithEmptyReplica, TestLeaderElectionZkExpiry, TestLeaderInitiatedRecoveryThread, TestOnReconnectListenerSupport, TestRebalanceLeaders, TestRequestForwarding, TestRequestStatusCollectionAPI, TestStressCloudBlindAtomicUpdates, AutoScalingHandlerTest, TestPolicyCloud, HdfsCollectionsAPIDistributedZkTest]
[junit4] Completed [569/724 (2!)] on J2 in 142.62s, 21 tests, 1 error <<< FAILURES!
[...truncated 43863 lines...]
[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1320 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1320/
2 tests failed.
FAILED: org.apache.lucene.index.TestIndexSorting.testRandom3
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([6E326D7A40C000D9]:0)
FAILED: junit.framework.TestSuite.org.apache.lucene.index.TestIndexSorting
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([6E326D7A40C000D9]:0)
Build Log:
[...truncated 1847 lines...]
[junit4] Suite: org.apache.lucene.index.TestIndexSorting
[junit4] 2> jun 09, 2017 1:58:50 PM com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
[junit4] 2> WARNING: Suite execution timed out: org.apache.lucene.index.TestIndexSorting
[junit4] 2> 1) Thread[id=8576, name=SUITE-TestIndexSorting-seed#[6E326D7A40C000D9], state=RUNNABLE, group=TGRP-TestIndexSorting]
[junit4] 2> at java.lang.Thread.getStackTrace(Thread.java:1556)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:696)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:693)
[junit4] 2> at java.security.AccessController.doPrivileged(Native Method)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getStackTrace(ThreadLeakControl.java:693)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getThreadsWithTraces(ThreadLeakControl.java:709)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:689)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:65)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:415)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:678)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:140)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:598)
[junit4] 2> 2) Thread[id=1, name=main, state=WAITING, group=main]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Thread.join(Thread.java:1249)
[junit4] 2> at java.lang.Thread.join(Thread.java:1323)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:608)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:457)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:244)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:355)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:13)
[junit4] 2> 3) Thread[id=9, name=JUnit4-serializer-daemon, state=TIMED_WAITING, group=main]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:50)
[junit4] 2> 4) Thread[id=8577, name=TEST-TestIndexSorting.testRandom3-seed#[6E326D7A40C000D9], state=RUNNABLE, group=TGRP-TestIndexSorting]
[junit4] 2> at sun.nio.ch.NativeThread.current(Native Method)
[junit4] 2> at sun.nio.ch.NativeThreadSet.add(NativeThreadSet.java:46)
[junit4] 2> at sun.nio.ch.FileChannelImpl.position(FileChannelImpl.java:284)
[junit4] 2> at sun.nio.ch.FileChannelImpl.position(FileChannelImpl.java:50)
[junit4] 2> at org.apache.lucene.mockfile.FilterSeekableByteChannel.position(FilterSeekableByteChannel.java:74)
[junit4] 2> at org.apache.lucene.mockfile.FilterSeekableByteChannel.position(FilterSeekableByteChannel.java:74)
[junit4] 2> at org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput.readInternal(SimpleFSDirectory.java:171)
[junit4] 2> at org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:342)
[junit4] 2> at org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:54)
[junit4] 2> at org.apache.lucene.store.DataInput.readInt(DataInput.java:102)
[junit4] 2> at org.apache.lucene.store.BufferedIndexInput.readInt(BufferedIndexInput.java:183)
[junit4] 2> at org.apache.lucene.store.MockIndexInputWrapper.readInt(MockIndexInputWrapper.java:165)
[junit4] 2> at org.apache.lucene.store.IndexInput$1.readInt(IndexInput.java:144)
[junit4] 2> at org.apache.lucene.util.packed.DirectReader$DirectPackedReader28.get(DirectReader.java:248)
[junit4] 2> at org.apache.lucene.codecs.lucene70.Lucene70DocValuesProducer$13.get(Lucene70DocValuesProducer.java:643)
[junit4] 2> at org.apache.lucene.codecs.lucene70.Lucene70DocValuesProducer$22.nextValue(Lucene70DocValuesProducer.java:1271)
[junit4] 2> at org.apache.lucene.codecs.DocValuesConsumer$3$1.nextValue(DocValuesConsumer.java:446)
[junit4] 2> at org.apache.lucene.codecs.lucene70.Lucene70DocValuesConsumer.writeValuesSingleBlock(Lucene70DocValuesConsumer.java:275)
[junit4] 2> at org.apache.lucene.codecs.lucene70.Lucene70DocValuesConsumer.writeValues(Lucene70DocValuesConsumer.java:263)
[junit4] 2> at org.apache.lucene.codecs.lucene70.Lucene70DocValuesConsumer.addSortedNumericField(Lucene70DocValuesConsumer.java:530)
[junit4] 2> at org.apache.lucene.codecs.DocValuesConsumer.mergeSortedNumericField(DocValuesConsumer.java:371)
[junit4] 2> at org.apache.lucene.codecs.DocValuesConsumer.merge(DocValuesConsumer.java:143)
[junit4] 2> at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.merge(PerFieldDocValuesFormat.java:151)
[junit4] 2> at org.apache.lucene.index.SegmentMerger.mergeDocValues(SegmentMerger.java:181)
[junit4] 2> at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:125)
[junit4] 2> at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4361)
[junit4] 2> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3933)
[junit4] 2> at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
[junit4] 2> at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2082)
[junit4] 2> at org.apache.lucene.index.IndexWriter.doAfterSegmentFlushed(IndexWriter.java:4992)
[junit4] 2> at org.apache.lucene.index.DocumentsWriter$MergePendingEvent.process(DocumentsWriter.java:731)
[junit4] 2> at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5030)
[junit4] 2> at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5021)
[junit4] 2> at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1573)
[junit4] 2> at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1315)
[junit4] 2> at org.apache.lucene.index.TestIndexSorting.testRandom3(TestIndexSorting.java:2301)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestIndexSorting -Dtests.method=testRandom3 -Dtests.seed=6E326D7A40C000D9 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=es-AR -Dtests.timezone=America/Indiana/Vevay -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 7200s J0 | TestIndexSorting.testRandom3 <<<
[junit4] > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
[junit4] > at __randomizedtesting.SeedInfo.seed([6E326D7A40C000D9]:0)
[junit4] 2> jun 09, 2017 1:58:50 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> jun 09, 2017 1:59:10 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.lucene.index.TestIndexSorting:
[junit4] 2> 1) Thread[id=8577, name=TEST-TestIndexSorting.testRandom3-seed#[6E326D7A40C000D9], state=RUNNABLE, group=TGRP-TestIndexSorting]
[junit4] 2> at org.apache.lucene.util.PriorityQueue.downHeap(PriorityQueue.java:276)
[junit4] 2> at org.apache.lucene.util.PriorityQueue.updateTop(PriorityQueue.java:211)
[junit4] 2> at org.apache.lucene.index.DocIDMerger$SortedDocIDMerger.next(DocIDMerger.java:188)
[junit4] 2> at org.apache.lucene.codecs.DocValuesConsumer$3$1.nextDoc(DocValuesConsumer.java:414)
[junit4] 2> at org.apache.lucene.codecs.LegacyDocValuesIterables$6$1.next(LegacyDocValuesIterables.java:295)
[junit4] 2> at org.apache.lucene.codecs.LegacyDocValuesIterables$6$1.next(LegacyDocValuesIterables.java:283)
[junit4] 2> at org.apache.lucene.codecs.memory.DirectDocValuesConsumer$1$1.next(DirectDocValuesConsumer.java:330)
[junit4] 2> at org.apache.lucene.codecs.memory.DirectDocValuesConsumer$1$1.next(DirectDocValuesConsumer.java:315)
[junit4] 2> at org.apache.lucene.codecs.memory.DirectDocValuesConsumer.addNumericFieldValues(DirectDocValuesConsumer.java:83)
[junit4] 2> at org.apache.lucene.codecs.memory.DirectDocValuesConsumer.addSortedNumericField(DirectDocValuesConsumer.java:262)
[junit4] 2> at org.apache.lucene.codecs.DocValuesConsumer.mergeSortedNumericField(DocValuesConsumer.java:371)
[junit4] 2> at org.apache.lucene.codecs.DocValuesConsumer.merge(DocValuesConsumer.java:143)
[junit4] 2> at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.merge(PerFieldDocValuesFormat.java:151)
[junit4] 2> at org.apache.lucene.index.SegmentMerger.mergeDocValues(SegmentMerger.java:181)
[junit4] 2> at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:125)
[junit4] 2> at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4361)
[junit4] 2> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3933)
[junit4] 2> at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
[junit4] 2> at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2082)
[junit4] 2> at org.apache.lucene.index.IndexWriter.doAfterSegmentFlushed(IndexWriter.java:4992)
[junit4] 2> at org.apache.lucene.index.DocumentsWriter$MergePendingEvent.process(DocumentsWriter.java:731)
[junit4] 2> at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5030)
[junit4] 2> at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5021)
[junit4] 2> at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1573)
[junit4] 2> at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1315)
[junit4] 2> at org.apache.lucene.index.TestIndexSorting.testRandom3(TestIndexSorting.java:2301)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> jun 09, 2017 1:59:10 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=8577, name=TEST-TestIndexSorting.testRandom3-seed#[6E326D7A40C000D9], state=RUNNABLE, group=TGRP-TestIndexSorting]
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/build/core/test/J0/temp/lucene.index.TestIndexSorting_6E326D7A40C000D9-001
[junit4] 2> Jun 09, 2017 5:59:10 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: All leaked threads terminated.
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {id=FSTOrd50}, docValues:{multi_valued_double=DocValuesFormat(name=Direct), multi_valued_long=DocValuesFormat(name=Asserting), bytes=DocValuesFormat(name=Direct), double=DocValuesFormat(name=Lucene70), foo=DocValuesFormat(name=Asserting), id=DocValuesFormat(name=Direct), multi_valued_int=DocValuesFormat(name=Lucene70), float=DocValuesFormat(name=Lucene70), multi_valued_bytes=DocValuesFormat(name=Lucene70), multi_valued_float=DocValuesFormat(name=Lucene70), int=DocValuesFormat(name=Direct), long=DocValuesFormat(name=Lucene70)}, maxPointsInLeafNode=437, maxMBSortInHeap=7.084589868215948, sim=RandomSimilarity(queryNorm=false): {}, locale=es-AR, timezone=America/Indiana/Vevay
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_121 (64-bit)/cpus=4,threads=1,free=168345616,total=333447168
[junit4] 2> NOTE: All tests run in this JVM: [TestDocValuesQueries, TestExitableDirectoryReader, TestTragicIndexWriterDeadlock, TestBM25Similarity, TestDocCount, TestWildcardRandom, TestBagOfPositions, TestCodecHoldsOpenFiles, TestCachingCollector, TestUniqueTermCount, TestBooleanMinShouldMatch, TestNewestSegment, TestFixedBitDocIdSet, LimitedFiniteStringsIteratorTest, TestIndexWriterExceptions, TestDocIDMerger, TestIndexWriterWithThreads, TestIndexWriterConfig, TestBinaryDocValuesUpdates, TestIndexWriterOutOfFileDescriptors, TestIndexWriter, TestIndexWriterNRTIsCurrent, TestPerSegmentDeletes, TestBlendedTermQuery, TestBooleanScorer, TestSnapshotDeletionPolicy, TestCodecs, TestBytesRef, TestForceMergeForever, TestSloppyPhraseQuery, TestSimilarityProvider, TestSPIClassIterator, TestMultiPhraseQuery, TestRateLimiter, TestNeverDelete, TestDateTools, TestTerm, TestConstantScoreQuery, TestNumericDocValuesUpdates, TestBlockPostingsFormat2, TestPerFieldPostingsFormat, TestTimSorter, TestSpanOrQuery, TestMaxPosition, TestOmitNorms, TestSimpleAttributeImpl, TestNamedSPILoader, TestSwappedIndexFiles, TestSmallFloat, TestMmapDirectory, TestNearSpansOrdered, TestIntroSorter, TestIntBlockPool, TestSimpleSearchEquivalence, Test4GBStoredFields, TestMixedDocValuesUpdates, TestNGramPhraseQuery, TestSpanContainQuery, TestFilterSpans, TestLucene70SegmentInfoFormat, TestGeoUtils, TestMSBRadixSorter, TestLucene50TermVectorsFormat, TestSingleInstanceLockFactory, TestBufferedChecksum, TestMultiThreadTermVectors, TestSearch, TestWordlistLoader, TestBoolean2, TestMultiPhraseEnum, TestMatchNoDocsQuery, TestFieldMaskingSpanQuery, TestRegExp, TestAutomatonQuery, TestDeletionPolicy, TestFlushByRamOrCountsPolicy, TestNRTThreads, TestTopDocsMerge, TestLongPostings, TestStressIndexing, TestRollingUpdates, TestTieredMergePolicy, TestIndexWriterForceMerge, TestLevenshteinAutomata, TestBufferedIndexInput, TestTermsEnum, TestConsistentFieldNumbers, TestSimpleExplanations, TestDirectoryReader, TestSegmentMerger, TestPhraseQuery, TestCollectionUtil, TestFixedBitSet, TestNRTCachingDirectory, TestFastDecompressionMode, TestPagedBytes, TestTransactions, TestCrash, TestSort, TestMultiDocValues, TestFieldsReader, TestDocValuesIndexing, TestSimilarityBase, TestHugeRamFile, TestDeterminizeLexicon, Test2BDocs, TestIndexWriterOnDiskFull, TestPostingsOffsets, TestTransactionRollback, TestFlex, TestUTF32ToUTF8, TestSpanExplanations, TestFastCompressionMode, TestLazyProxSkipping, TestDocsAndPositions, TestMergeSchedulerExternal, TestBagOfPostings, TestTermVectorsWriter, TestBytesRefHash, TestThreadedForceMerge, TestStressNRT, TestIndexInput, TestOmitTf, TestSameScoresWithThreads, TestMultiFields, TestTermRangeQuery, TestDocumentsWriterDeleteQueue, TestDocumentWriter, TestCrashCausesCorruptIndex, TestMultiTermConstantScore, TestLiveFieldValues, TestForTooMuchCloning, TestSpanSearchEquivalence, TestIndexedDISI, TestGeoEncodingUtils, TestPolygon, TestPolygon2D, Test2BPoints, Test2BSortedDocValuesOrds, TestAllFilesCheckIndexHeader, TestExceedMaxTermLength, TestFieldInvertState, TestFilterCodecReader, TestFilterDirectoryReader, TestIndexSorting]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestIndexSorting -Dtests.seed=6E326D7A40C000D9 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=es-AR -Dtests.timezone=America/Indiana/Vevay -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J0 | TestIndexSorting (suite) <<<
[junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4] > at __randomizedtesting.SeedInfo.seed([6E326D7A40C000D9]:0)
[junit4] Completed [453/453 (1!)] on J0 in 7220.15s, 3 tests, 2 errors <<< FAILURES!
[...truncated 55296 lines...]
[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1319 - Still
unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1319/
1 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.hdfs.StressHdfsTest
Error Message:
ObjectTracker found 2 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347) at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:361) at org.apache.solr.core.SolrCore.cleanupOldIndexDirectories(SolrCore.java:3015) at org.apache.solr.core.SolrCore.close(SolrCore.java:1579) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:994) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:855) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:979) at org.apache.solr.core.CoreContainer.lambda$load$7(CoreContainer.java:611) at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347) at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:361) at org.apache.solr.core.SolrCore.cleanupOldIndexDirectories(SolrCore.java:3015) at org.apache.solr.core.SolrCore.close(SolrCore.java:1579) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:994) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:855) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:979) at org.apache.solr.core.CoreContainer.lambda$load$7(CoreContainer.java:611) at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)
Stack Trace:
java.lang.AssertionError: ObjectTracker found 2 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:361)
at org.apache.solr.core.SolrCore.cleanupOldIndexDirectories(SolrCore.java:3015)
at org.apache.solr.core.SolrCore.close(SolrCore.java:1579)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:994)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:855)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:979)
at org.apache.solr.core.CoreContainer.lambda$load$7(CoreContainer.java:611)
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:361)
at org.apache.solr.core.SolrCore.cleanupOldIndexDirectories(SolrCore.java:3015)
at org.apache.solr.core.SolrCore.close(SolrCore.java:1579)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:994)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:855)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:979)
at org.apache.solr.core.CoreContainer.lambda$load$7(CoreContainer.java:611)
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([5FC01C2D33532281]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:295)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 13178 lines...]
[junit4] Suite: org.apache.solr.cloud.hdfs.StressHdfsTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_5FC01C2D33532281-001/init-core-data-001
[junit4] 2> 4360669 WARN (SUITE-StressHdfsTest-seed#[5FC01C2D33532281]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=42 numCloses=42
[junit4] 2> 4360669 INFO (SUITE-StressHdfsTest-seed#[5FC01C2D33532281]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields
[junit4] 2> 4360671 INFO (SUITE-StressHdfsTest-seed#[5FC01C2D33532281]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
[junit4] 2> 4360671 INFO (SUITE-StressHdfsTest-seed#[5FC01C2D33532281]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 4360722 WARN (SUITE-StressHdfsTest-seed#[5FC01C2D33532281]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 4360730 WARN (SUITE-StressHdfsTest-seed#[5FC01C2D33532281]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 4360732 INFO (SUITE-StressHdfsTest-seed#[5FC01C2D33532281]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 4360749 INFO (SUITE-StressHdfsTest-seed#[5FC01C2D33532281]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_33276_hdfs____7d5m8r/webapp
[junit4] 2> 4362146 INFO (SUITE-StressHdfsTest-seed#[5FC01C2D33532281]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33276
[junit4] 2> 4362232 WARN (SUITE-StressHdfsTest-seed#[5FC01C2D33532281]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 4362233 INFO (SUITE-StressHdfsTest-seed#[5FC01C2D33532281]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 4362245 INFO (SUITE-StressHdfsTest-seed#[5FC01C2D33532281]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_56136_datanode____qmkl8a/webapp
[junit4] 2> 4362636 INFO (SUITE-StressHdfsTest-seed#[5FC01C2D33532281]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:56136
[junit4] 2> 4362697 WARN (SUITE-StressHdfsTest-seed#[5FC01C2D33532281]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 4362699 INFO (SUITE-StressHdfsTest-seed#[5FC01C2D33532281]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 4362712 INFO (SUITE-StressHdfsTest-seed#[5FC01C2D33532281]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_40313_datanode____.60gphc/webapp
[junit4] 2> 4362755 INFO (IPC Server handler 4 on 44131) [ ] BlockStateChange BLOCK* processReport: from storage DS-f3a4ed9f-f395-4fba-a1a2-3bbe54989399 node DatanodeRegistration(127.0.0.1:57767, datanodeUuid=46cdda94-49d7-4e8f-b067-8c27c24fbe79, infoPort=40243, infoSecurePort=0, ipcPort=43596, storageInfo=lv=-56;cid=testClusterID;nsid=391903592;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 4362756 INFO (IPC Server handler 4 on 44131) [ ] BlockStateChange BLOCK* processReport: from storage DS-299a82f0-dd14-49ef-a458-9d15cf1185e5 node DatanodeRegistration(127.0.0.1:57767, datanodeUuid=46cdda94-49d7-4e8f-b067-8c27c24fbe79, infoPort=40243, infoSecurePort=0, ipcPort=43596, storageInfo=lv=-56;cid=testClusterID;nsid=391903592;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 4363143 INFO (SUITE-StressHdfsTest-seed#[5FC01C2D33532281]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40313
[junit4] 2> 4363254 INFO (IPC Server handler 2 on 44131) [ ] BlockStateChange BLOCK* processReport: from storage DS-8166f56d-0b2a-4b08-b1ce-1c8f72c2f077 node DatanodeRegistration(127.0.0.1:60595, datanodeUuid=89f31a2d-c4db-4469-91e3-5428c308cbe3, infoPort=55399, infoSecurePort=0, ipcPort=52404, storageInfo=lv=-56;cid=testClusterID;nsid=391903592;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 4363255 INFO (IPC Server handler 2 on 44131) [ ] BlockStateChange BLOCK* processReport: from storage DS-5a89b1b7-79a7-4e5b-b22a-75f2a0f7a5d4 node DatanodeRegistration(127.0.0.1:60595, datanodeUuid=89f31a2d-c4db-4469-91e3-5428c308cbe3, infoPort=55399, infoSecurePort=0, ipcPort=52404, storageInfo=lv=-56;cid=testClusterID;nsid=391903592;c=0), blocks: 0, hasStaleStorage: false, processing time: 1 msecs
[junit4] 2> 4363314 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 4363314 INFO (Thread-31519) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 4363314 INFO (Thread-31519) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 4363316 ERROR (Thread-31519) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 4363414 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.c.ZkTestServer start zk server on port:45162
[junit4] 2> 4363420 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15c889873fc0000, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 4363427 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 4363429 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 4363431 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 4363432 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 4363434 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 4363435 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 4363437 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 4363438 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 4363440 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 4363441 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 4363443 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 4363446 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use TLOG replicas unless explicitly asked otherwise
[junit4] 2> 4363559 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_5FC01C2D33532281-001/control-001/cores/collection1
[junit4] 2> 4363560 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 4363564 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7c57277e{/,null,AVAILABLE}
[junit4] 2> 4363566 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1704b7a8{HTTP/1.1,[http/1.1]}{127.0.0.1:33100}
[junit4] 2> 4363566 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.e.j.s.Server Started @4367094ms
[junit4] 2> 4363566 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:44131/hdfs__localhost_44131__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-master_checkout_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.StressHdfsTest_5FC01C2D33532281-001_tempDir-002_control_data, replicaType=NRT, hostContext=/, hostPort=33100, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_5FC01C2D33532281-001/control-001/cores}
[junit4] 2> 4363566 ERROR (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 4363566 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 4363566 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 4363567 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 4363574 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-06-08T16:45:11.806Z
[junit4] 2> 4363577 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 4363577 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_5FC01C2D33532281-001/control-001/solr.xml
[junit4] 2> 4363582 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@58887ce9, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 4363586 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45162/solr
[junit4] 2> 4363613 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [n:127.0.0.1:33100_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 4363614 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [n:127.0.0.1:33100_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:33100_
[junit4] 2> 4363615 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [n:127.0.0.1:33100_ ] o.a.s.c.Overseer Overseer (id=98103480257609732-127.0.0.1:33100_-n_0000000000) starting
[junit4] 2> 4363623 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [n:127.0.0.1:33100_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33100_
[junit4] 2> 4363628 INFO (zkCallback-30338-thread-1-processing-n:127.0.0.1:33100_) [n:127.0.0.1:33100_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 4363698 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [n:127.0.0.1:33100_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58887ce9
[junit4] 2> 4363705 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [n:127.0.0.1:33100_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58887ce9
[junit4] 2> 4363705 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [n:127.0.0.1:33100_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58887ce9
[junit4] 2> 4363707 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [n:127.0.0.1:33100_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_5FC01C2D33532281-001/control-001/cores
[junit4] 2> 4363707 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [n:127.0.0.1:33100_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 4363708 INFO (coreLoadExecutor-16650-thread-1-processing-n:127.0.0.1:33100_) [n:127.0.0.1:33100_ c:control_collection x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 4363715 INFO (OverseerStateUpdate-98103480257609732-127.0.0.1:33100_-n_0000000000) [n:127.0.0.1:33100_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 4363732 INFO (coreLoadExecutor-16650-thread-1-processing-n:127.0.0.1:33100_) [n:127.0.0.1:33100_ c:control_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 4363759 INFO (coreLoadExecutor-16650-thread-1-processing-n:127.0.0.1:33100_) [n:127.0.0.1:33100_ c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 4363884 INFO (coreLoadExecutor-16650-thread-1-processing-n:127.0.0.1:33100_) [n:127.0.0.1:33100_ c:control_collection x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 4363902 INFO (coreLoadExecutor-16650-thread-1-processing-n:127.0.0.1:33100_) [n:127.0.0.1:33100_ c:control_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection, trusted=true
[junit4] 2> 4363902 INFO (coreLoadExecutor-16650-thread-1-processing-n:127.0.0.1:33100_) [n:127.0.0.1:33100_ c:control_collection x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.core_node1' (registry 'solr.core.control_collection.shard1.core_node1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58887ce9
[junit4] 2> 4363902 INFO (coreLoadExecutor-16650-thread-1-processing-n:127.0.0.1:33100_) [n:127.0.0.1:33100_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:44131/solr_hdfs_home
[junit4] 2> 4363902 INFO (coreLoadExecutor-16650-thread-1-processing-n:127.0.0.1:33100_) [n:127.0.0.1:33100_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 4363902 INFO (coreLoadExecutor-16650-thread-1-processing-n:127.0.0.1:33100_) [n:127.0.0.1:33100_ c:control_collection x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 4363902 INFO (coreLoadExecutor-16650-thread-1-processing-n:127.0.0.1:33100_) [n:127.0.0.1:33100_ c:control_collection x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_5FC01C2D33532281-001/control-001/cores/collection1], dataDir=[hdfs://localhost:44131/solr_hdfs_home/control_collection/core_node1/data/]
[junit4] 2> 4363904 INFO (coreLoadExecutor-16650-thread-1-processing-n:127.0.0.1:33100_) [n:127.0.0.1:33100_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:44131/solr_hdfs_home/control_collection/core_node1/data/snapshot_metadata
[junit4] 2> 4363911 INFO (coreLoadExecutor-16650-thread-1-processing-n:127.0.0.1:33100_) [n:127.0.0.1:33100_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 4363911 INFO (coreLoadExecutor-16650-thread-1-processing-n:127.0.0.1:33100_) [n:127.0.0.1:33100_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 4363912 INFO (coreLoadExecutor-16650-thread-1-processing-n:127.0.0.1:33100_) [n:127.0.0.1:33100_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 4363918 INFO (coreLoadExecutor-16650-thread-1-processing-n:127.0.0.1:33100_) [n:127.0.0.1:33100_ c:control_collection x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 4363919 INFO (coreLoadExecutor-16650-thread-1-processing-n:127.0.0.1:33100_) [n:127.0.0.1:33100_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:44131/solr_hdfs_home/control_collection/core_node1/data
[junit4] 2> 4363935 INFO (coreLoadExecutor-16650-thread-1-processing-n:127.0.0.1:33100_) [n:127.0.0.1:33100_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:44131/solr_hdfs_home/control_collection/core_node1/data/index
[junit4] 2> 4363941 INFO (coreLoadExecutor-16650-thread-1-processing-n:127.0.0.1:33100_) [n:127.0.0.1:33100_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 4363941 INFO (coreLoadExecutor-16650-thread-1-processing-n:127.0.0.1:33100_) [n:127.0.0.1:33100_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 4363941 INFO (coreLoadExecutor-16650-thread-1-processing-n:127.0.0.1:33100_) [n:127.0.0.1:33100_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 4363947 INFO (coreLoadExecutor-16650-thread-1-processing-n:127.0.0.1:33100_) [n:127.0.0.1:33100_ c:control_collection x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 4363947 INFO (coreLoadExecutor-16650-thread-1-processing-n:127.0.0.1:33100_) [n:127.0.0.1:33100_ c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=19, maxMergeAtOnceExplicit=25, maxMergedSegmentMB=70.68359375, floorSegmentMB=0.595703125, forceMergeDeletesPctAllowed=9.843885925992309, segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.32485238773878367
[junit4] 2> 4363959 INFO (IPC Server handler 6 on 44131) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:57767 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-8166f56d-0b2a-4b08-b1ce-1c8f72c2f077:NORMAL:127.0.0.1:60595|RBW], ReplicaUC[[DISK]DS-f3a4ed9f-f395-4fba-a1a2-3bbe54989399:NORMAL:127.0.0.1:57767|FINALIZED]]} size 0
[junit4] 2> 4363960 INFO (IPC Server handler 7 on 44131) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:60595 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-f3a4ed9f-f395-4fba-a1a2-3bbe54989399:NORMAL:127.0.0.1:57767|FINALIZED], ReplicaUC[[DISK]DS-5a89b1b7-79a7-4e5b-b22a-75f2a0f7a5d4:NORMAL:127.0.0.1:60595|FINALIZED]]} size 0
[junit4] 2> 4363964 WARN (coreLoadExecutor-16650-thread-1-processing-n:127.0.0.1:33100_) [n:127.0.0.1:33100_ c:control_collection x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 4364012 INFO (coreLoadExecutor-16650-thread-1-processing-n:127.0.0.1:33100_) [n:127.0.0.1:33100_ c:control_collection x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 4364012 INFO (coreLoadExecutor-16650-thread-1-processing-n:127.0.0.1:33100_) [n:127.0.0.1:33100_ c:control_collection x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 4364012 INFO (coreLoadExecutor-16650-thread-1-processing-n:127.0.0.1:33100_) [n:127.0.0.1:33100_ c:control_collection x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 4364022 INFO (coreLoadExecutor-16650-thread-1-processing-n:127.0.0.1:33100_) [n:127.0.0.1:33100_ c:control_collection x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 4364022 INFO (coreLoadExecutor-16650-thread-1-processing-n:127.0.0.1:33100_) [n:127.0.0.1:33100_ c:control_collection x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 4364024 INFO (coreLoadExecutor-16650-thread-1-processing-n:127.0.0.1:33100_) [n:127.0.0.1:33100_ c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=668559482, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 4364028 INFO (coreLoadExecutor-16650-thread-1-processing-n:127.0.0.1:33100_) [n:127.0.0.1:33100_ c:control_collection x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@57fc6a41[collection1] main]
[junit4] 2> 4364029 INFO (coreLoadExecutor-16650-thread-1-processing-n:127.0.0.1:33100_) [n:127.0.0.1:33100_ c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 4364030 INFO (coreLoadExecutor-16650-thread-1-processing-n:127.0.0.1:33100_) [n:127.0.0.1:33100_ c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 4364030 INFO (coreLoadExecutor-16650-thread-1-processing-n:127.0.0.1:33100_) [n:127.0.0.1:33100_ c:control_collection x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 4364032 INFO (searcherExecutor-16651-thread-1-processing-n:127.0.0.1:33100_ x:collection1 c:control_collection) [n:127.0.0.1:33100_ c:control_collection x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@57fc6a41[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 4364032 INFO (coreLoadExecutor-16650-thread-1-processing-n:127.0.0.1:33100_) [n:127.0.0.1:33100_ c:control_collection x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1569655684873584640
[junit4] 2> 4364038 INFO (coreZkRegister-16643-thread-1-processing-n:127.0.0.1:33100_ x:collection1 c:control_collection) [n:127.0.0.1:33100_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 4364038 INFO (coreZkRegister-16643-thread-1-processing-n:127.0.0.1:33100_ x:collection1 c:control_collection) [n:127.0.0.1:33100_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 4364038 INFO (coreZkRegister-16643-thread-1-processing-n:127.0.0.1:33100_ x:collection1 c:control_collection) [n:127.0.0.1:33100_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:33100/collection1/
[junit4] 2> 4364038 INFO (coreZkRegister-16643-thread-1-processing-n:127.0.0.1:33100_ x:collection1 c:control_collection) [n:127.0.0.1:33100_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 4364038 INFO (coreZkRegister-16643-thread-1-processing-n:127.0.0.1:33100_ x:collection1 c:control_collection) [n:127.0.0.1:33100_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:33100/collection1/ has no replicas
[junit4] 2> 4364039 INFO (coreZkRegister-16643-thread-1-processing-n:127.0.0.1:33100_ x:collection1 c:control_collection) [n:127.0.0.1:33100_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 4364042 INFO (coreZkRegister-16643-thread-1-processing-n:127.0.0.1:33100_ x:collection1 c:control_collection) [n:127.0.0.1:33100_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:33100/collection1/ shard1
[junit4] 2> 4364193 INFO (coreZkRegister-16643-thread-1-processing-n:127.0.0.1:33100_ x:collection1 c:control_collection) [n:127.0.0.1:33100_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 4364231 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 4364232 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:45162/solr ready
[junit4] 2> 4364232 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 4364232 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
[junit4] 2> 4364345 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_5FC01C2D33532281-001/shard-1-001/cores/collection1
[junit4] 2> 4364346 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_5FC01C2D33532281-001/shard-1-001 of type TLOG
[junit4] 2> 4364347 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 4364349 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4275dcef{/,null,AVAILABLE}
[junit4] 2> 4364350 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@434a2bc6{HTTP/1.1,[http/1.1]}{127.0.0.1:45981}
[junit4] 2> 4364350 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.e.j.s.Server Started @4367878ms
[junit4] 2> 4364350 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:44131/hdfs__localhost_44131__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-master_checkout_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.StressHdfsTest_5FC01C2D33532281-001_tempDir-002_jetty1, replicaType=TLOG, solrconfig=solrconfig.xml, hostContext=/, hostPort=45981, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_5FC01C2D33532281-001/shard-1-001/cores}
[junit4] 2> 4364350 ERROR (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 4364351 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 4364351 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 4364351 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 4364351 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-06-08T16:45:12.584Z
[junit4] 2> 4364355 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15c889873fc0007, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 4364355 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 4364355 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_5FC01C2D33532281-001/shard-1-001/solr.xml
[junit4] 2> 4364361 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@58887ce9, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 4364365 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45162/solr
[junit4] 2> 4364377 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [n:127.0.0.1:45981_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 4364381 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [n:127.0.0.1:45981_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 4364383 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [n:127.0.0.1:45981_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45981_
[junit4] 2> 4364384 INFO (zkCallback-30342-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 4364385 INFO (zkCallback-30338-thread-2-processing-n:127.0.0.1:33100_) [n:127.0.0.1:33100_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 4364386 INFO (zkCallback-30348-thread-1-processing-n:127.0.0.1:45981_) [n:127.0.0.1:45981_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 4364462 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [n:127.0.0.1:45981_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58887ce9
[junit4] 2> 4364469 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [n:127.0.0.1:45981_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58887ce9
[junit4] 2> 4364470 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [n:127.0.0.1:45981_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58887ce9
[junit4] 2> 4364473 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [n:127.0.0.1:45981_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_5FC01C2D33532281-001/shard-1-001/cores
[junit4] 2> 4364473 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [n:127.0.0.1:45981_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 4364474 INFO (coreLoadExecutor-16661-thread-1-processing-n:127.0.0.1:45981_) [n:127.0.0.1:45981_ c:collection1 x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 4364476 INFO (OverseerStateUpdate-98103480257609732-127.0.0.1:33100_-n_0000000000) [n:127.0.0.1:33100_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 4364578 INFO (zkCallback-30348-thread-1-processing-n:127.0.0.1:45981_) [n:127.0.0.1:45981_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 4365494 INFO (coreLoadExecutor-16661-thread-1-processing-n:127.0.0.1:45981_) [n:127.0.0.1:45981_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 4365509 INFO (coreLoadExecutor-16661-thread-1-processing-n:127.0.0.1:45981_) [n:127.0.0.1:45981_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 4365635 INFO (coreLoadExecutor-16661-thread-1-processing-n:127.0.0.1:45981_) [n:127.0.0.1:45981_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 4365649 INFO (coreLoadExecutor-16661-thread-1-processing-n:127.0.0.1:45981_) [n:127.0.0.1:45981_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1, trusted=true
[junit4] 2> 4365649 INFO (coreLoadExecutor-16661-thread-1-processing-n:127.0.0.1:45981_) [n:127.0.0.1:45981_ c:collection1 x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.core_node1' (registry 'solr.core.collection1.shard1.core_node1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58887ce9
[junit4] 2> 4365649 INFO (coreLoadExecutor-16661-thread-1-processing-n:127.0.0.1:45981_) [n:127.0.0.1:45981_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:44131/solr_hdfs_home
[junit4] 2> 4365649 INFO (coreLoadExecutor-16661-thread-1-processing-n:127.0.0.1:45981_) [n:127.0.0.1:45981_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 4365650 INFO (coreLoadExecutor-16661-thread-1-processing-n:127.0.0.1:45981_) [n:127.0.0.1:45981_ c:collection1 x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 4365650 INFO (coreLoadExecutor-16661-thread-1-processing-n:127.0.0.1:45981_) [n:127.0.0.1:45981_ c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_5FC01C2D33532281-001/shard-1-001/cores/collection1], dataDir=[hdfs://localhost:44131/solr_hdfs_home/collection1/core_node1/data/]
[junit4] 2> 4365651 INFO (coreLoadExecutor-16661-thread-1-processing-n:127.0.0.1:45981_) [n:127.0.0.1:45981_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:44131/solr_hdfs_home/collection1/core_node1/data/snapshot_metadata
[junit4] 2> 4365671 INFO (coreLoadExecutor-16661-thread-1-processing-n:127.0.0.1:45981_) [n:127.0.0.1:45981_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 4365671 INFO (coreLoadExecutor-16661-thread-1-processing-n:127.0.0.1:45981_) [n:127.0.0.1:45981_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 4365671 INFO (coreLoadExecutor-16661-thread-1-processing-n:127.0.0.1:45981_) [n:127.0.0.1:45981_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 4366849 INFO (coreLoadExecutor-16661-thread-1-processing-n:127.0.0.1:45981_) [n:127.0.0.1:45981_ c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@440b8e59
[junit4] 2> 4366849 INFO (coreLoadExecutor-16661-thread-1-processing-n:127.0.0.1:45981_) [n:127.0.0.1:45981_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:44131/solr_hdfs_home/collection1/core_node1/data
[junit4] 2> 4366859 INFO (coreLoadExecutor-16661-thread-1-processing-n:127.0.0.1:45981_) [n:127.0.0.1:45981_ c:collection1 x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1.shard1.core_node1, tag=1141608025
[junit4] 2> 4366860 INFO (coreLoadExecutor-16661-thread-1-processing-n:127.0.0.1:45981_) [n:127.0.0.1:45981_ c:collection1 x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard1.leader, tag=1141608025
[junit4] 2> 4366860 ERROR (coreLoadExecutor-16661-thread-1-processing-n:127.0.0.1:45981_) [n:127.0.0.1:45981_ c:collection1 x:collection1] o.a.s.c.SolrCore Error while closing
[junit4] 2> java.lang.NullPointerException
[junit4] 2> at org.apache.solr.core.SolrCore.close(SolrCore.java:1604)
[junit4] 2> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:994)
[junit4] 2> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:855)
[junit4] 2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:979)
[junit4] 2> at org.apache.solr.core.CoreContainer.lambda$load$7(CoreContainer.java:611)
[junit4] 2> at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 4366861 ERROR (coreContainerWorkExecutor-16657-thread-1-processing-n:127.0.0.1:45981_) [n:127.0.0.1:45981_ ] o.a.s.c.CoreContainer Error waiting for SolrCore to be created
[junit4] 2> java.util.concurrent.ExecutionException: org.apache.solr.common.SolrException: Unable to create core [collection1]
[junit4] 2> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
[junit4] 2> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
[junit4] 2> at org.apache.solr.core.CoreContainer.lambda$load$8(CoreContainer.java:639)
[junit4] 2> at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> Caused by: org.apache.solr.common.SolrException: Unable to create core [collection1]
[junit4] 2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:994)
[junit4] 2> at org.apache.solr.core.CoreContainer.lambda$load$7(CoreContainer.java:611)
[junit4] 2> at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
[junit4] 2> ... 5 more
[junit4] 2> Caused by: org.apache.solr.common.SolrException: The max direct memory is likely too low. Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~8.388608 MB.
[junit4] 2> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1002)
[junit4] 2> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:855)
[junit4] 2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:979)
[junit4] 2> ... 7 more
[junit4] 2> Caused by: java.lang.RuntimeException: The max direct memory is likely too low. Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~8.388608 MB.
[junit4] 2> at org.apache.solr.core.HdfsDirectoryFactory.createBlockCache(HdfsDirectoryFactory.java:311)
[junit4] 2> at org.apache.solr.core.HdfsDirectoryFactory.getBlockDirectoryCache(HdfsDirectoryFactory.java:287)
[junit4] 2> at org.apache.solr.core.HdfsDirectoryFactory.create(HdfsDirectoryFactory.java:227)
[junit4] 2> at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:346)
[junit4] 2> at org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:490)
[junit4] 2> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:942)
[junit4] 2> ... 9 more
[junit4] 2> Caused by: java.lang.OutOfMemoryError: Direct buffer memory
[junit4] 2> at java.nio.Bits.reserveMemory(Bits.java:693)
[junit4] 2> at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
[junit4] 2> at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
[junit4] 2> at org.apache.solr.store.blockcache.BlockCache.<init>(BlockCache.java:71)
[junit4] 2> at org.apache.solr.core.HdfsDirectoryFactory.createBlockCache(HdfsDirectoryFactory.java:309)
[junit4] 2> ... 14 more
[junit4] 2> 4367449 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_5FC01C2D33532281-001/shard-2-001/cores/collection1
[junit4] 2> 4367450 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_5FC01C2D33532281-001/shard-2-001 of type TLOG
[junit4] 2> 4367451 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 4367452 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@115e0238{/,null,AVAILABLE}
[junit4] 2> 4367453 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@4922b8a8{HTTP/1.1,[http/1.1]}{127.0.0.1:43980}
[junit4] 2> 4367454 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.e.j.s.Server Started @4370982ms
[junit4] 2> 4367454 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:44131/hdfs__localhost_44131__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-master_checkout_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.StressHdfsTest_5FC01C2D33532281-001_tempDir-002_jetty2, replicaType=TLOG, solrconfig=solrconfig.xml, hostContext=/, hostPort=43980, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_5FC01C2D33532281-001/shard-2-001/cores}
[junit4] 2> 4367455 ERROR (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 4367455 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 4367455 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 4367455 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 4367455 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-06-08T16:45:15.688Z
[junit4] 2> 4367458 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 4367458 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_5FC01C2D33532281-001/shard-2-001/solr.xml
[junit4] 2> 4367463 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@58887ce9, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 4367470 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45162/solr
[junit4] 2> 4367481 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [n:127.0.0.1:43980_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 4367485 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [n:127.0.0.1:43980_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 4367490 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [n:127.0.0.1:43980_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43980_
[junit4] 2> 4367493 INFO (zkCallback-30348-thread-1-processing-n:127.0.0.1:45981_) [n:127.0.0.1:45981_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 4367493 INFO (zkCallback-30342-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 4367494 INFO (zkCallback-30354-thread-1-processing-n:127.0.0.1:43980_) [n:127.0.0.1:43980_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 4367502 INFO (zkCallback-30338-thread-2-processing-n:127.0.0.1:33100_) [n:127.0.0.1:33100_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 4367568 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [n:127.0.0.1:43980_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58887ce9
[junit4] 2> 4367576 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [n:127.0.0.1:43980_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58887ce9
[junit4] 2> 4367576 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [n:127.0.0.1:43980_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58887ce9
[junit4] 2> 4367579 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [n:127.0.0.1:43980_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_5FC01C2D33532281-001/shard-2-001/cores
[junit4] 2> 4367579 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [n:127.0.0.1:43980_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 4367580 INFO (coreLoadExecutor-16668-thread-1-processing-n:127.0.0.1:43980_) [n:127.0.0.1:43980_ c:collection1 x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 4367581 INFO (OverseerStateUpdate-98103480257609732-127.0.0.1:33100_-n_0000000000) [n:127.0.0.1:33100_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 4367582 INFO (zkCallback-30348-thread-1-processing-n:127.0.0.1:45981_) [n:127.0.0.1:45981_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 4367604 INFO (coreLoadExecutor-16668-thread-1-processing-n:127.0.0.1:43980_) [n:127.0.0.1:43980_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 4367619 INFO (coreLoadExecutor-16668-thread-1-processing-n:127.0.0.1:43980_) [n:127.0.0.1:43980_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 4367744 INFO (coreLoadExecutor-16668-thread-1-processing-n:127.0.0.1:43980_) [n:127.0.0.1:43980_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 4367761 INFO (coreLoadExecutor-16668-thread-1-processing-n:127.0.0.1:43980_) [n:127.0.0.1:43980_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1, trusted=true
[junit4] 2> 4367761 INFO (coreLoadExecutor-16668-thread-1-processing-n:127.0.0.1:43980_) [n:127.0.0.1:43980_ c:collection1 x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.core_node2' (registry 'solr.core.collection1.shard1.core_node2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58887ce9
[junit4] 2> 4367762 INFO (coreLoadExecutor-16668-thread-1-processing-n:127.0.0.1:43980_) [n:127.0.0.1:43980_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:44131/solr_hdfs_home
[junit4] 2> 4367762 INFO (coreLoadExecutor-16668-thread-1-processing-n:127.0.0.1:43980_) [n:127.0.0.1:43980_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 4367762 INFO (coreLoadExecutor-16668-thread-1-processing-n:127.0.0.1:43980_) [n:127.0.0.1:43980_ c:collection1 x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 4367762 INFO (coreLoadExecutor-16668-thread-1-processing-n:127.0.0.1:43980_) [n:127.0.0.1:43980_ c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_5FC01C2D33532281-001/shard-2-001/cores/collection1], dataDir=[hdfs://localhost:44131/solr_hdfs_home/collection1/core_node2/data/]
[junit4] 2> 4367763 INFO (coreLoadExecutor-16668-thread-1-processing-n:127.0.0.1:43980_) [n:127.0.0.1:43980_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:44131/solr_hdfs_home/collection1/core_node2/data/snapshot_metadata
[junit4] 2> 4367771 INFO (coreLoadExecutor-16668-thread-1-processing-n:127.0.0.1:43980_) [n:127.0.0.1:43980_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 4367771 INFO (coreLoadExecutor-16668-thread-1-processing-n:127.0.0.1:43980_) [n:127.0.0.1:43980_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 4367771 INFO (coreLoadExecutor-16668-thread-1-processing-n:127.0.0.1:43980_) [n:127.0.0.1:43980_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 4368034 INFO (coreLoadExecutor-16668-thread-1-processing-n:127.0.0.1:43980_) [n:127.0.0.1:43980_ c:collection1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 4368035 INFO (coreLoadExecutor-16668-thread-1-processing-n:127.0.0.1:43980_) [n:127.0.0.1:43980_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:44131/solr_hdfs_home/collection1/core_node2/data
[junit4] 2> 4368056 INFO (coreLoadExecutor-16668-thread-1-processing-n:127.0.0.1:43980_) [n:127.0.0.1:43980_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:44131/solr_hdfs_home/collection1/core_node2/data/index
[junit4] 2> 4368062 INFO (coreLoadExecutor-16668-thread-1-processing-n:127.0.0.1:43980_) [n:127.0.0.1:43980_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 4368062 INFO (coreLoadExecutor-16668-thread-1-processing-n:127.0.0.1:43980_) [n:127.0.0.1:43980_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 4368062 INFO (coreLoadExecutor-16668-thread-1-processing-n:127.0.0.1:43980_) [n:127.0.0.1:43980_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 4368066 INFO (coreLoadExecutor-16668-thread-1-processing-n:127.0.0.1:43980_) [n:127.0.0.1:43980_ c:collection1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 4368066 INFO (coreLoadExecutor-16668-thread-1-processing-n:127.0.0.1:43980_) [n:127.0.0.1:43980_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=19, maxMergeAtOnceExplicit=25, maxMergedSegmentMB=70.68359375, floorSegmentMB=0.595703125, forceMergeDeletesPctAllowed=9.843885925992309, segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.32485238773878367
[junit4] 2> 4368077 INFO (IPC Server handler 5 on 44131) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:57767 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-5a89b1b7-79a7-4e5b-b22a-75f2a0f7a5d4:NORMAL:127.0.0.1:60595|RBW], ReplicaUC[[DISK]DS-299a82f0-dd14-49ef-a458-9d15cf1185e5:NORMAL:127.0.0.1:57767|RBW]]} size 0
[junit4] 2> 4368078 INFO (IPC Server handler 9 on 44131) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:60595 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-299a82f0-dd14-49ef-a458-9d15cf1185e5:NORMAL:127.0.0.1:57767|RBW], ReplicaUC[[DISK]DS-8166f56d-0b2a-4b08-b1ce-1c8f72c2f077:NORMAL:127.0.0.1:60595|FINALIZED]]} size 0
[junit4] 2> 4368083 WARN (coreLoadExecutor-16668-thread-1-processing-n:127.0.0.1:43980_) [n:127.0.0.1:43980_ c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 4368130 INFO (coreLoadExecutor-16668-thread-1-processing-n:127.0.0.1:43980_) [n:127.0.0.1:43980_ c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 4368130 INFO (coreLoadExecutor-16668-thread-1-processing-n:127.0.0.1:43980_) [n:127.0.0.1:43980_ c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 4368130 INFO (coreLoadExecutor-16668-thread-1-processing-n:127.0.0.1:43980_) [n:127.0.0.1:43980_ c:collection1 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 4368141 INFO (coreLoadExecutor-16668-thread-1-processing-n:127.0.0.1:43980_) [n:127.0.0.1:43980_ c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 4368141 INFO (coreLoadExecutor-16668-thread-1-processing-n:127.0.0.1:43980_) [n:127.0.0.1:43980_ c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 4368144 INFO (coreLoadExecutor-16668-thread-1-processing-n:127.0.0.1:43980_) [n:127.0.0.1:43980_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=668559482, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 4368148 INFO (coreLoadExecutor-16668-thread-1-processing-n:127.0.0.1:43980_) [n:127.0.0.1:43980_ c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@7da0c83e[collection1] main]
[junit4] 2> 4368149 INFO (coreLoadExecutor-16668-thread-1-processing-n:127.0.0.1:43980_) [n:127.0.0.1:43980_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 4368150 INFO (coreLoadExecutor-16668-thread-1-processing-n:127.0.0.1:43980_) [n:127.0.0.1:43980_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 4368151 INFO (coreLoadExecutor-16668-thread-1-processing-n:127.0.0.1:43980_) [n:127.0.0.1:43980_ c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 4368152 INFO (searcherExecutor-16669-thread-1-processing-n:127.0.0.1:43980_ x:collection1 c:collection1) [n:127.0.0.1:43980_ c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@7da0c83e[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 4368152 INFO (coreLoadExecutor-16668-thread-1-processing-n:127.0.0.1:43980_) [n:127.0.0.1:43980_ c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1569655689193717760
[junit4] 2> 4368159 INFO (coreZkRegister-16663-thread-1-processing-n:127.0.0.1:43980_ x:collection1 c:collection1) [n:127.0.0.1:43980_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9999ms
[junit4] 2> 4368261 INFO (zkCallback-30348-thread-1-processing-n:127.0.0.1:45981_) [n:127.0.0.1:45981_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 4368261 INFO (zkCallback-30354-thread-1-processing-n:127.0.0.1:43980_) [n:127.0.0.1:43980_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 4368731 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_5FC01C2D33532281-001/shard-3-001/cores/collection1
[junit4] 2> 4368732 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_5FC01C2D33532281-001/shard-3-001 of type TLOG
[junit4] 2> 4368735 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 4368737 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@19da04ed{/,null,AVAILABLE}
[junit4] 2> 4368738 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@32d6f878{HTTP/1.1,[http/1.1]}{127.0.0.1:37716}
[junit4] 2> 4368738 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.e.j.s.Server Started @4372266ms
[junit4] 2> 4368738 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:44131/hdfs__localhost_44131__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-master_checkout_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.StressHdfsTest_5FC01C2D33532281-001_tempDir-002_jetty3, replicaType=TLOG, solrconfig=solrconfig.xml, hostContext=/, hostPort=37716, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_5FC01C2D33532281-001/shard-3-001/cores}
[junit4] 2> 4368739 ERROR (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 4368739 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 4368739 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 4368739 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 4368739 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-06-08T16:45:16.972Z
[junit4] 2> 4368742 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 4368742 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_5FC01C2D33532281-001/shard-3-001/solr.xml
[junit4] 2> 4368749 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@58887ce9, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 4368752 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45162/solr
[junit4] 2> 4368769 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [n:127.0.0.1:37716_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 4368771 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [n:127.0.0.1:37716_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 4368773 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [n:127.0.0.1:37716_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37716_
[junit4] 2> 4368775 INFO (zkCallback-30342-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 4368775 INFO (zkCallback-30348-thread-1-processing-n:127.0.0.1:45981_) [n:127.0.0.1:45981_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 4368775 INFO (zkCallback-30354-thread-1-processing-n:127.0.0.1:43980_) [n:127.0.0.1:43980_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 4368775 INFO (zkCallback-30338-thread-2-processing-n:127.0.0.1:33100_) [n:127.0.0.1:33100_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 4368776 INFO (zkCallback-30360-thread-1-processing-n:127.0.0.1:37716_) [n:127.0.0.1:37716_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 4368909 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [n:127.0.0.1:37716_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58887ce9
[junit4] 2> 4368916 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [n:127.0.0.1:37716_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58887ce9
[junit4] 2> 4368917 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [n:127.0.0.1:37716_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58887ce9
[junit4] 2> 4368919 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [n:127.0.0.1:37716_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_5FC01C2D33532281-001/shard-3-001/cores
[junit4] 2> 4368919 INFO (TEST-StressHdfsTest.test-seed#[5FC01C2D33532281]) [n:127.0.0.1:37716_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 4368920 INFO (coreLoadExecutor-16679-thread-1-processing-n:127.0.0.1:37716_) [n:127.0.0.1:37716_ c:collection1 x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 4368921 INFO (OverseerStateUpdate-98103480257609732-127.0.0.1:33100_-n_0000000000) [n:127.0.0.1:33100_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 4369024 INFO (zkCallback-30360-thread-1-processing-n:127.0.0.1:37716_) [n:127.0.0.1:37716_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 4369024 INFO (zkCallback-30348-thread-1-processing-n:127.0.0.1:45981_) [n:127.0.0.1:45981_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 4369024 INFO (zkCallback-30354-thread-1-processing-n:127.0.0.1:43980_) [n:127.0.0.1:43980_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 4369934 INFO (coreLoadExecutor-16679-thread-1-processing-n:127.0.0.1:37716_) [n:127.0.0.1:37716_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 4369949 INFO (coreLoadExecutor-16679-thread-1-processing-n:127.0.0.1:37716_) [n:127.0.0.1:37716_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 4370077 INFO (coreLoadExecutor-16679-thread-1-processing-n:127.0.0.1:37716_) [n:127.0.0.1:37716_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 4370090 INFO (coreLoadExecutor-16679-thread-1-processing-n:127.0.0.1:37716_) [n:127.0.0.1:37716_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1, trusted=true
[junit4] 2> 4370090 INFO (coreLoadExecutor-16679-thread-1-processing-n:127.0.0.1:37716_) [n:127.0.0.1:37716_ c:collection1 x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.core_node3' (registry 'solr.core.collection1.shard1.core_node3') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58887ce9
[junit4] 2> 4370090 INFO (coreLoadExecutor-16679-thread-1-processing-n:127.0.0.1:37716_) [n:127.0.0.1:37716_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:44131/solr_hdfs_home
[junit4] 2> 4370090 INFO (coreLoadExecutor-16679-thread-1-processing-n:127.0.0.1:37716_) [n:127.0.0.1:37716_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 4370090 INFO (coreLoadExecutor-16679-thread-1-processing-n:127.0.0.1:37716_) [n:127.0.0.1:37716_ c:collection1 x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 4370090 INFO (coreLoadExecutor-16679-thread-1-processing-n:127.0.0.1:37716_) [n:127.0.0.1:37716_ c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_5FC01C2D33532281-001/shard-3-001/cores/collection1], dataDir=[hdfs://localhost:44131/solr_hdfs_home/collection1/core_node3/data/]
[junit4] 2> 4370092 INFO (coreLoadExecutor-16679-thread-1-processing-n:127.0.0.1:37716_) [n:127.0.0.1:37716_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:44131/solr_hdfs_home/collection1/core_node3/data/snapshot_metadata
[junit4] 2> 4370099 INFO (coreLoadExecutor-16679-thread-1-processing-n:127.0.0.1:37716_) [n:127.0.0.1:37716_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 4370099 INFO (coreLoadExecutor-16679-thread-1-processing-n:127.0.0.1:37716_) [n:127.0.0.1:37716_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 4370099 INFO (coreLoadExecutor-16679-thread-1-processing-n:127.0.0.1:37716_) [n:127.0.0.1:37716_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 4370104 INFO (coreLoadExecutor-16679-thread-1-processing-n:127.0.0.1:37716_) [n:127.0.0.1:37716_ c:collection1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 4370104 INFO (coreLoadExecutor-16679-thread-1-processing-n:127.0.0.1:37716_) [n:127.0.0.1:37716_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:44131/solr_hdfs_home/collection1/core_node3/data
[junit4] 2> 4370122 INFO (coreLoadExecutor-16679-thread-1-processing-n:127.0.0.1:37716_) [n:127.0.0.1:37716_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:44131/solr_hdfs_home/collection1/core_node3/data/index
[junit4] 2> 4370128 INFO (coreLoadExecutor-16679-thread-1-processing-n:127.0.0.1:37716_) [n:127.0.0.1:37716_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 4370128 INFO (coreLoadExecutor-16679-thread-1-processing-n:127.0.0.1:37716_) [n:127.0.0.1:37716_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 4370128 INFO (coreLoadExecutor-16679-thread-1-processing-n:127.0.0.1:37716_) [n:127.0.0.1:37716_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 4370132 INFO (coreLoadExecutor-16679-thread-1-processing-n:127.0.0.1:37716_) [n:127.0.0.1:37716_ c:collection1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 4370132 INFO (coreLoadExecutor-16679-thread-1-processing-n:127.0.0.1:37716_) [n:127.0.0.1:37716_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=19, maxMergeAtOnceExplicit=25, maxMergedSegmentMB=70.68359375, floorSegmentMB=0.595703125, forceMergeDeletesPctAllowed=9.843885925992309, segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.32485238773878367
[junit4] 2> 4370140 INFO (IPC Server handler 7 on 44131) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:57767 is added to blk_1073741827_1003{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-5a89b1b7-79a7-4e5b-b22a-75f2a0f7a5d4:NORMAL:127.0.0.1:60595|RBW], ReplicaUC[[DISK]DS-f3a4ed9f-f395-4fba-a1a2-3bbe54989399:NORMAL:127.0.0.1:57767|FINALIZED]]} size 0
[junit4] 2> 4370140 INFO (IPC Server handler 6 on 44131) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:60595 is added to blk_1073741827_1003{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-5a89b1b7-79a7-4e5b-b22a-75f2a0f7a5d4:NORMAL:127.0.0.1:60595|RBW], ReplicaUC[[DISK]DS-f3a4ed9f-f395-4fba-a1a2-3bbe54989399:NORMAL:127.0.0.1:57767|FINALIZED]]} size 0
[junit4] 2> 4370145 WARN (coreLoadExecutor-16679-thread-1-processing-n:127.0.0.1:37716_) [n:127.0.0.1:37716_ c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 4370192 INFO (coreLoadExecutor-16679-thread-1-processing-n:127.0.0.1:37716_) [n:127.0.0.1:37716_ c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 4370192 INFO (coreLoadExecutor-16679-thread-1-processing-n:127.0.0.1:37716_) [n:127.0.0.1:37716_ c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 4370192 INFO (coreLoadExecutor-16679-thread-1-processing-n:127.0.0.1:37716_) [n:127.0.0.1:37716_ c:collection1 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 4370215 INFO (coreLoadExecutor-16679-thread-1-processing-n:127.0.0.1:37716_) [n:127.0.0.1:37716_ c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 4370215 INFO (coreLoadExecutor-16679-thread-1-processing-n:127.0.0.1:37716_) [n:127.0.0.1:37716_ c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 4370217 INFO (coreLoadExecutor-16679-thread-1-processing-n:127.0.0.1:37716_) [n:127.0.0.1:37716_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=668559482, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 4370220 INFO (coreLoadExecutor-16679-thread-1-processing-n:127.0.0.1:37716_) [n:127.0.0.1:37716_ c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2aaf8111[collection1] main]
[junit4] 2> 4370222 INFO (coreLoadExecutor-16679-thread-1-processing-n:127.0.0.1:37716_) [n:127.0.0.1:37716_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 4370222 INFO (coreLoadExecutor-16679-thread-1-processing-n:127.0.0.1:37716_) [n:127.0.0.1:37716_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 4370223 INFO (coreLoadExecutor-16679-thread-1-processing-n:127.0.0.1:37716_) [n:127.0.0.1:37716_ c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 4370224 INFO (searcherExecutor-16680-thread-1-processing-n:127.0.0.1:37716_ x:collection1 c:collection1) [n:127.0.0.1:37716_ c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2aaf8111[collection1]
[...truncated too long message...]
cet_i_dv=PostingsFormat(name=LuceneFixedGap), text=PostingsFormat(name=MockRandom), range_facet_l=PostingsFormat(name=LuceneFixedGap), timestamp=PostingsFormat(name=LuceneFixedGap), txt_t=PostingsFormat(name=MockRandom)}, docValues:{range_facet_l_dv=DocValuesFormat(name=Memory), _version_=DocValuesFormat(name=Lucene70), multiDefault=DocValuesFormat(name=Direct), intDefault=DocValuesFormat(name=Lucene70), range_facet_i_dv=DocValuesFormat(name=Lucene70), id=DocValuesFormat(name=Memory), text=DocValuesFormat(name=Asserting), intDvoDefault=DocValuesFormat(name=Asserting), timestamp=DocValuesFormat(name=Lucene70), range_facet_l=DocValuesFormat(name=Lucene70), txt_t=DocValuesFormat(name=Asserting)}, maxPointsInLeafNode=58, maxMBSortInHeap=6.238577090193301, sim=RandomSimilarity(queryNorm=true): {}, locale=lv-LV, timezone=Asia/Sakhalin
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_121 (64-bit)/cpus=4,threads=2,free=228200376,total=526385152
[junit4] 2> NOTE: All tests run in this JVM: [SpatialHeatmapFacetsTest, TestRandomDVFaceting, SolrSlf4jReporterTest, FastVectorHighlighterTest, TestBadConfig, TestLegacyFieldCache, TestTolerantUpdateProcessorRandomCloud, RequiredFieldsTest, TestQuerySenderListener, OverriddenZkACLAndCredentialsProvidersTest, SpellCheckCollatorWithCollapseTest, TestStressLiveNodes, TestPseudoReturnFields, PathHierarchyTokenizerFactoryTest, PeerSyncReplicationTest, TestUninvertingReader, TestLegacyTerms, TestTestInjection, CursorPagingTest, TestFieldCache, CurrencyFieldOpenExchangeTest, StandardRequestHandlerTest, BasicZkTest, TestCollectionAPIs, TestSubQueryTransformer, DefaultValueUpdateProcessorTest, HighlighterMaxOffsetTest, ResponseLogComponentTest, InfixSuggestersTest, HdfsDirectoryTest, TestStressLucene, CreateCollectionCleanupTest, TestRealTimeGet, SolrMetricManagerTest, TestFilteredDocIdSet, TestSolr4Spatial, CdcrUpdateLogTest, SpellingQueryConverterTest, TestLazyCores, SmileWriterTest, AnalysisAfterCoreReloadTest, TestOmitPositions, TestLMJelinekMercerSimilarityFactory, SubstringBytesRefFilterTest, QueryElevationComponentTest, NotRequiredUniqueKeyTest, SearchHandlerTest, ConfigureRecoveryStrategyTest, TestSystemCollAutoCreate, PreAnalyzedFieldTest, TestSortByMinMaxFunction, TestAddFieldRealTimeGet, TestFastWriter, TestBackupRepositoryFactory, SolrRequestParserTest, TestExactStatsCache, TestLFUCache, TestSuggestSpellingConverter, UniqFieldsUpdateProcessorFactoryTest, OverseerRolesTest, TestSolrDeletionPolicy2, TestDelegationWithHadoopAuth, TestReplicationHandler, RegexBoostProcessorTest, ConcurrentDeleteAndCreateCollectionTest, TestMissingGroups, TestAnalyzedSuggestions, TestBulkSchemaConcurrent, TestQueryWrapperFilter, HdfsBasicDistributedZk2Test, SolrIndexMetricsTest, TestSolrConfigHandler, SignatureUpdateProcessorFactoryTest, ChaosMonkeySafeLeaderWithPullReplicasTest, AtomicUpdateProcessorFactoryTest, CacheHeaderTest, TestExportWriter, TestCloudInspectUtil, TestReRankQParserPlugin, DistributedQueryElevationComponentTest, UninvertDocValuesMergePolicyTest, ReturnFieldsTest, TestRangeQuery, QueryEqualityTest, DateMathParserTest, TestUseDocValuesAsStored, HttpPartitionTest, HttpSolrCallGetCoreTest, LeaderFailureAfterFreshStartTest, BasicAuthStandaloneTest, TestDocumentBuilder, CopyFieldTest, JvmMetricsTest, TestHighlightDedupGrouping, TestNamedUpdateProcessors, BlockCacheTest, UpdateLogTest, TestDistributedStatsComponentCardinality, DistributedSpellCheckComponentTest, ImplicitSnitchTest, DocExpirationUpdateProcessorFactoryTest, TestFieldResource, TestFileDictionaryLookup, TestSolrCoreSnapshots, TestJmxIntegration, CdcrRequestHandlerTest, SolrXmlInZkTest, DeleteShardTest, PreAnalyzedUpdateProcessorTest, SimpleCollectionCreateDeleteTest, TestPivotHelperCode, TestBulkSchemaAPI, ReplaceNodeTest, SyncSliceTest, FileBasedSpellCheckerTest, UUIDFieldTest, TestDFRSimilarityFactory, TestConfigReload, TestCloudNestedDocsSort, ShardRoutingTest, TestRandomFaceting, TestZkChroot, ZkCLITest, AutoCommitTest, TestFaceting, TestHashPartitioner, TestJoin, SolrCoreTest, TestGroupingSearch, SolrCmdDistributorTest, PeerSyncTest, TestSort, TestFiltering, TestFunctionQuery, SoftAutoCommitTest, ShowFileRequestHandlerTest, CurrencyFieldXmlFileTest, SolrIndexSplitterTest, SimplePostToolTest, TestCoreDiscovery, SpellCheckCollatorTest, CoreAdminHandlerTest, TestFoldingMultitermQuery, DocValuesTest, SpatialFilterTest, TestCSVLoader, PolyFieldTest, SchemaVersionSpecificBehaviorTest, SolrCoreCheckLockOnStartupTest, TestAtomicUpdateErrorCases, SortByFunctionTest, TestRemoteStreaming, DocValuesMultiTest, TestSolrDeletionPolicy1, SolrInfoBeanTest, XsltUpdateRequestHandlerTest, DebugComponentTest, TestSurroundQueryParser, DisMaxRequestHandlerTest, DirectSolrSpellCheckerTest, TermsComponentTest, DocumentBuilderTest, TermVectorComponentTest, MoreLikeThisHandlerTest, IndexSchemaRuntimeFieldTest, TestCSVResponseWriter, TestPHPSerializedResponseWriter, CSVRequestHandlerTest, TestMergePolicyConfig, MultiTermTest, SampleTest, TestDocSet, TestBinaryField, TestSearchPerf, NumericFieldsTest, MinimalSchemaTest, TestElisionMultitermQuery, TestFuzzyAnalyzedSuggestions, TestSolrCoreProperties, SpellPossibilityIteratorTest, TestCodecSupport, ChaosMonkeyNothingIsSafeTest, CdcrReplicationDistributedZkTest, ChaosMonkeyNothingIsSafeWithPullReplicasTest, CollectionsAPIAsyncDistributedZkTest, ConfigSetsAPITest, CustomCollectionTest, DeleteReplicaTest, DistribDocExpirationUpdateProcessorTest, DocValuesNotIndexedTest, LeaderElectionContextKeyTest, LeaderFailoverAfterPartitionTest, MissingSegmentRecoveryTest, MultiThreadedOCPTest, RollingRestartTest, TestTolerantUpdateProcessorCloud, TlogReplayBufferedWhileIndexingTest, HdfsRecoverLeaseTest, HdfsRecoveryZkTest, HdfsSyncSliceTest, HdfsUnloadDistributedZkTest, StressHdfsTest]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=StressHdfsTest -Dtests.seed=5FC01C2D33532281 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=lv-LV -Dtests.timezone=Asia/Sakhalin -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 0.00s J0 | StressHdfsTest (suite) <<<
[junit4] > Throwable #1: java.lang.AssertionError: ObjectTracker found 2 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory]
[junit4] > org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory
[junit4] > at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
[junit4] > at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
[junit4] > at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:361)
[junit4] > at org.apache.solr.core.SolrCore.cleanupOldIndexDirectories(SolrCore.java:3015)
[junit4] > at org.apache.solr.core.SolrCore.close(SolrCore.java:1579)
[junit4] > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:994)
[junit4] > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:855)
[junit4] > at org.apache.solr.core.CoreContainer.create(CoreContainer.java:979)
[junit4] > at org.apache.solr.core.CoreContainer.lambda$load$7(CoreContainer.java:611)
[junit4] > at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
[junit4] > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] > at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory
[junit4] > at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
[junit4] > at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
[junit4] > at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:361)
[junit4] > at org.apache.solr.core.SolrCore.cleanupOldIndexDirectories(SolrCore.java:3015)
[junit4] > at org.apache.solr.core.SolrCore.close(SolrCore.java:1579)
[junit4] > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:994)
[junit4] > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:855)
[junit4] > at org.apache.solr.core.CoreContainer.create(CoreContainer.java:979)
[junit4] > at org.apache.solr.core.CoreContainer.lambda$load$7(CoreContainer.java:611)
[junit4] > at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
[junit4] > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] > at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > at __randomizedtesting.SeedInfo.seed([5FC01C2D33532281]:0)
[junit4] > at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:295)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] Completed [562/725 (1!)] on J0 in 92.24s, 1 test, 1 failure <<< FAILURES!
[...truncated 43899 lines...]
[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1318 - Failure
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1318/
2 tests failed.
FAILED: org.apache.lucene.index.TestIndexSorting.testRandom3
Error Message:
Java heap space
Stack Trace:
java.lang.OutOfMemoryError: Java heap space
at __randomizedtesting.SeedInfo.seed([B1C4C4B25A7D350:A9C402914155FA56]:0)
at org.apache.lucene.util.packed.Packed8ThreeBlocks.<init>(Packed8ThreeBlocks.java:41)
at org.apache.lucene.util.packed.PackedInts.getMutable(PackedInts.java:963)
at org.apache.lucene.util.packed.PackedInts.getMutable(PackedInts.java:939)
at org.apache.lucene.util.packed.GrowableWriter.ensureCapacity(GrowableWriter.java:80)
at org.apache.lucene.util.packed.GrowableWriter.set(GrowableWriter.java:88)
at org.apache.lucene.util.packed.AbstractPagedMutable.set(AbstractPagedMutable.java:98)
at org.apache.lucene.util.fst.NodeHash.addNew(NodeHash.java:152)
at org.apache.lucene.util.fst.NodeHash.rehash(NodeHash.java:169)
at org.apache.lucene.util.fst.NodeHash.add(NodeHash.java:133)
at org.apache.lucene.util.fst.Builder.compileNode(Builder.java:200)
at org.apache.lucene.util.fst.Builder.freezeTail(Builder.java:296)
at org.apache.lucene.util.fst.Builder.add(Builder.java:400)
at org.apache.lucene.codecs.memory.MemoryDocValuesConsumer.writeFST(MemoryDocValuesConsumer.java:373)
at org.apache.lucene.codecs.memory.MemoryDocValuesConsumer.addSortedField(MemoryDocValuesConsumer.java:416)
at org.apache.lucene.codecs.memory.MemoryDocValuesConsumer.addSortedField(MemoryDocValuesConsumer.java:406)
at org.apache.lucene.codecs.DocValuesConsumer.mergeSortedField(DocValuesConsumer.java:527)
at org.apache.lucene.codecs.DocValuesConsumer.merge(DocValuesConsumer.java:139)
at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.merge(PerFieldDocValuesFormat.java:151)
at org.apache.lucene.index.SegmentMerger.mergeDocValues(SegmentMerger.java:181)
at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:125)
at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4361)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3933)
at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2082)
at org.apache.lucene.index.IndexWriter.doAfterSegmentFlushed(IndexWriter.java:4992)
at org.apache.lucene.index.DocumentsWriter$MergePendingEvent.process(DocumentsWriter.java:731)
at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5030)
at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5021)
at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1573)
at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1315)
at org.apache.lucene.index.TestIndexSorting.testRandom3(TestIndexSorting.java:2301)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
FAILED: org.apache.solr.cloud.hdfs.HdfsRecoveryZkTest.test
Error Message:
Mismatch in counts between replicas
Stack Trace:
java.lang.AssertionError: Mismatch in counts between replicas
at __randomizedtesting.SeedInfo.seed([19F448DC1133CDE:89CB7B576FEF5126]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.RecoveryZkTest.assertShardConsistency(RecoveryZkTest.java:143)
at org.apache.solr.cloud.RecoveryZkTest.test(RecoveryZkTest.java:126)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 1535 lines...]
[junit4] Suite: org.apache.lucene.index.TestIndexSorting
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestIndexSorting -Dtests.method=testRandom3 -Dtests.seed=B1C4C4B25A7D350 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=id -Dtests.timezone=Africa/Bangui -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 149s J1 | TestIndexSorting.testRandom3 <<<
[junit4] > Throwable #1: java.lang.OutOfMemoryError: Java heap space
[junit4] > at __randomizedtesting.SeedInfo.seed([B1C4C4B25A7D350:A9C402914155FA56]:0)
[junit4] > at org.apache.lucene.util.packed.Packed8ThreeBlocks.<init>(Packed8ThreeBlocks.java:41)
[junit4] > at org.apache.lucene.util.packed.PackedInts.getMutable(PackedInts.java:963)
[junit4] > at org.apache.lucene.util.packed.PackedInts.getMutable(PackedInts.java:939)
[junit4] > at org.apache.lucene.util.packed.GrowableWriter.ensureCapacity(GrowableWriter.java:80)
[junit4] > at org.apache.lucene.util.packed.GrowableWriter.set(GrowableWriter.java:88)
[junit4] > at org.apache.lucene.util.packed.AbstractPagedMutable.set(AbstractPagedMutable.java:98)
[junit4] > at org.apache.lucene.util.fst.NodeHash.addNew(NodeHash.java:152)
[junit4] > at org.apache.lucene.util.fst.NodeHash.rehash(NodeHash.java:169)
[junit4] > at org.apache.lucene.util.fst.NodeHash.add(NodeHash.java:133)
[junit4] > at org.apache.lucene.util.fst.Builder.compileNode(Builder.java:200)
[junit4] > at org.apache.lucene.util.fst.Builder.freezeTail(Builder.java:296)
[junit4] > at org.apache.lucene.util.fst.Builder.add(Builder.java:400)
[junit4] > at org.apache.lucene.codecs.memory.MemoryDocValuesConsumer.writeFST(MemoryDocValuesConsumer.java:373)
[junit4] > at org.apache.lucene.codecs.memory.MemoryDocValuesConsumer.addSortedField(MemoryDocValuesConsumer.java:416)
[junit4] > at org.apache.lucene.codecs.memory.MemoryDocValuesConsumer.addSortedField(MemoryDocValuesConsumer.java:406)
[junit4] > at org.apache.lucene.codecs.DocValuesConsumer.mergeSortedField(DocValuesConsumer.java:527)
[junit4] > at org.apache.lucene.codecs.DocValuesConsumer.merge(DocValuesConsumer.java:139)
[junit4] > at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.merge(PerFieldDocValuesFormat.java:151)
[junit4] > at org.apache.lucene.index.SegmentMerger.mergeDocValues(SegmentMerger.java:181)
[junit4] > at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:125)
[junit4] > at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4361)
[junit4] > at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3933)
[junit4] > at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
[junit4] > at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2082)
[junit4] > at org.apache.lucene.index.IndexWriter.doAfterSegmentFlushed(IndexWriter.java:4992)
[junit4] > at org.apache.lucene.index.DocumentsWriter$MergePendingEvent.process(DocumentsWriter.java:731)
[junit4] > at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5030)
[junit4] > at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5021)
[junit4] > at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1573)
[junit4] > at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1315)
[junit4] > at org.apache.lucene.index.TestIndexSorting.testRandom3(TestIndexSorting.java:2301)
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/build/core/test/J1/temp/lucene.index.TestIndexSorting_B1C4C4B25A7D350-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {sparse_text=PostingsFormat(name=LuceneVarGapFixedInterval), docs=BlockTreeOrds(blocksize=128), norms=BlockTreeOrds(blocksize=128), positions=PostingsFormat(name=Memory), id=BlockTreeOrds(blocksize=128), term_vectors=BlockTreeOrds(blocksize=128)}, docValues:{multi_valued_long=DocValuesFormat(name=Lucene70), foo=DocValuesFormat(name=Lucene70), numeric=DocValuesFormat(name=Memory), multi_valued_numeric=DocValuesFormat(name=Memory), float=DocValuesFormat(name=Direct), long=DocValuesFormat(name=Direct), points=DocValuesFormat(name=Lucene70), sparse_int=DocValuesFormat(name=Direct), bar=DocValuesFormat(name=Memory), multi_valued_double=DocValuesFormat(name=Memory), docs=DocValuesFormat(name=Memory), id=DocValuesFormat(name=Memory), multi_valued_int=DocValuesFormat(name=Lucene70), multi_valued_bytes=DocValuesFormat(name=Memory), sparse_text=DocValuesFormat(name=Direct), double=DocValuesFormat(name=Memory), sparse_binary=DocValuesFormat(name=Direct), positions=DocValuesFormat(name=Lucene70), int=DocValuesFormat(name=Memory), sorted=DocValuesFormat(name=Memory), sparse=DocValuesFormat(name=Direct), multi_valued_string=DocValuesFormat(name=Memory), norms=DocValuesFormat(name=Memory), bytes=DocValuesFormat(name=Memory), dense_int=DocValuesFormat(name=Memory), binary=DocValuesFormat(name=Memory), multi_valued_float=DocValuesFormat(name=Lucene70), term_vectors=DocValuesFormat(name=Memory)}, maxPointsInLeafNode=538, maxMBSortInHeap=6.188537060958199, sim=RandomSimilarity(queryNorm=true): {positions=DFR I(ne)BZ(0.3), id=IB SPL-DZ(0.3), term_vectors=LM Jelinek-Mercer(0.700000)}, locale=id, timezone=Africa/Bangui
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_121 (64-bit)/cpus=4,threads=1,free=362264744,total=427819008
[junit4] 2> NOTE: All tests run in this JVM: [TestMinimize, TestPositiveScoresOnlyCollector, TestDuelingCodecs, TestFSTs, TestOneMergeWrappingMergePolicy, TestDocumentsWriterDeleteQueue, TestPhrasePrefixQuery, TestTermdocPerf, TestTieredMergePolicy, TestOmitPositions, TestQueryRescorer, TestSpanFirstQuery, TestPhraseQuery, TestFieldInvertState, TestPrefixInBooleanQuery, TestLSBRadixSorter, TestAddIndexes, TestMinShouldMatch2, TestIndexSearcher, TestBooleanOr, TestGraphTokenizers, TestAllFilesDetectTruncation, TestSimpleFSDirectory, TestSearcherManager, TestElevationComparator, TestTermVectors, TestPositionIncrement, TestIndexWriterDelete, TestIndexWriterOnJRECrash, TestPriorityQueue, TestIndexWriterMaxDocs, Test2BBKDPoints, TestRollback, TestNoMergeScheduler, TestLucene50StoredFieldsFormat, TestDocInverterPerFieldErrorInfo, TestVirtualMethod, TestSpanTermQuery, TestMergePolicyWrapper, Test2BPostingsBytes, TestSpanExplanationsOfNonMatches, TestScoreCachingWrappingScorer, TestSleepingLockWrapper, TestCodecUtil, TestIndexWriterUnicode, TestFieldType, TestNorms, TestNumericUtils, TestRadixSelector, TestSortRescorer, TestCloseableThreadLocal, TestSizeBoundedForceMerge, TestPolygon, TestSpanBoostQuery, TestSameScoresWithThreads, TestLevenshteinAutomata, TestLogMergePolicy, TestFilterDirectoryReader, TestOmitTf, TestReusableStringReader, TestCheckIndex, TestDocumentsWriterStallControl, Test2BDocs, TestEarlyTerminatingSortingCollector, TestSortRandom, TestDuelingCodecsAtNight, TestParallelReaderEmptyIndex, TestLucene70DocValuesFormat, TestDirectMonotonic, TestIndexWriterDeleteByQuery, TestMultiTermConstantScore, TestDemo, TestRecyclingIntBlockAllocator, TestTryDelete, TestMultiDocValues, TestDocIDMerger, TestIndexSorting]
[junit4] Completed [394/453 (1!)] on J1 in 178.96s, 50 tests, 1 error <<< FAILURES!
[...truncated 195 lines...]
[junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/lucene/build/core/test/temp/junit4-J1-20170607_223721_4833503895863574774592.sysout
[junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
[junit4] java.lang.OutOfMemoryError: Java heap space
[junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/heapdumps/java_pid31990.hprof ...
[junit4] Heap dump file created [454543972 bytes in 1.734 secs]
[junit4] <<< JVM J1: EOF ----
[...truncated 11471 lines...]
[junit4] Suite: org.apache.solr.cloud.hdfs.HdfsRecoveryZkTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_19F448DC1133CDE-001/init-core-data-001
[junit4] 2> 3394314 INFO (SUITE-HdfsRecoveryZkTest-seed#[19F448DC1133CDE]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields
[junit4] 2> 3394322 INFO (SUITE-HdfsRecoveryZkTest-seed#[19F448DC1133CDE]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 3394322 INFO (SUITE-HdfsRecoveryZkTest-seed#[19F448DC1133CDE]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_19F448DC1133CDE-001/tempDir-001
[junit4] 2> 3394322 INFO (SUITE-HdfsRecoveryZkTest-seed#[19F448DC1133CDE]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 3394323 INFO (Thread-14473) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 3394323 INFO (Thread-14473) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 3394325 ERROR (Thread-14473) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 3394423 INFO (SUITE-HdfsRecoveryZkTest-seed#[19F448DC1133CDE]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:39024
[junit4] 2> 3394445 INFO (jetty-launcher-3606-thread-1) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 3394445 INFO (jetty-launcher-3606-thread-2) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 3394446 INFO (jetty-launcher-3606-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2e8d383e{/solr,null,AVAILABLE}
[junit4] 2> 3394448 INFO (jetty-launcher-3606-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6ff6afc1{/solr,null,AVAILABLE}
[junit4] 2> 3394448 INFO (jetty-launcher-3606-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@48764c05{HTTP/1.1,[http/1.1]}{127.0.0.1:34679}
[junit4] 2> 3394449 INFO (jetty-launcher-3606-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1b1eba3a{HTTP/1.1,[http/1.1]}{127.0.0.1:34068}
[junit4] 2> 3394449 INFO (jetty-launcher-3606-thread-2) [ ] o.e.j.s.Server Started @3397233ms
[junit4] 2> 3394449 INFO (jetty-launcher-3606-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=34068}
[junit4] 2> 3394449 INFO (jetty-launcher-3606-thread-1) [ ] o.e.j.s.Server Started @3397233ms
[junit4] 2> 3394449 INFO (jetty-launcher-3606-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=34679}
[junit4] 2> 3394449 ERROR (jetty-launcher-3606-thread-2) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3394449 ERROR (jetty-launcher-3606-thread-1) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3394449 INFO (jetty-launcher-3606-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.0.0
[junit4] 2> 3394449 INFO (jetty-launcher-3606-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.0.0
[junit4] 2> 3394449 INFO (jetty-launcher-3606-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3394449 INFO (jetty-launcher-3606-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3394449 INFO (jetty-launcher-3606-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3394449 INFO (jetty-launcher-3606-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3394450 INFO (jetty-launcher-3606-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-06-08T02:45:04.198Z
[junit4] 2> 3394450 INFO (jetty-launcher-3606-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-06-08T02:45:04.198Z
[junit4] 2> 3394462 INFO (jetty-launcher-3606-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3394462 INFO (jetty-launcher-3606-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3394476 INFO (jetty-launcher-3606-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39024/solr
[junit4] 2> 3394479 INFO (jetty-launcher-3606-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39024/solr
[junit4] 2> 3394521 INFO (jetty-launcher-3606-thread-1) [n:127.0.0.1:34679_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3394521 INFO (jetty-launcher-3606-thread-2) [n:127.0.0.1:34068_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3394522 INFO (jetty-launcher-3606-thread-1) [n:127.0.0.1:34679_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:34679_solr
[junit4] 2> 3394523 INFO (jetty-launcher-3606-thread-2) [n:127.0.0.1:34068_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34068_solr
[junit4] 2> 3394525 INFO (jetty-launcher-3606-thread-1) [n:127.0.0.1:34679_solr ] o.a.s.c.Overseer Overseer (id=98100176753262598-127.0.0.1:34679_solr-n_0000000000) starting
[junit4] 2> 3394531 INFO (zkCallback-3618-thread-1-processing-n:127.0.0.1:34679_solr) [n:127.0.0.1:34679_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3394532 INFO (zkCallback-3617-thread-1-processing-n:127.0.0.1:34068_solr) [n:127.0.0.1:34068_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3394569 INFO (jetty-launcher-3606-thread-1) [n:127.0.0.1:34679_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34679_solr
[junit4] 2> 3394579 INFO (zkCallback-3618-thread-1-processing-n:127.0.0.1:34679_solr) [n:127.0.0.1:34679_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 3394579 INFO (zkCallback-3617-thread-1-processing-n:127.0.0.1:34068_solr) [n:127.0.0.1:34068_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 3394693 INFO (jetty-launcher-3606-thread-1) [n:127.0.0.1:34679_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34679.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@74421fac
[junit4] 2> 3394697 INFO (jetty-launcher-3606-thread-1) [n:127.0.0.1:34679_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34679.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@74421fac
[junit4] 2> 3394698 INFO (jetty-launcher-3606-thread-1) [n:127.0.0.1:34679_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34679.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@74421fac
[junit4] 2> 3394699 INFO (jetty-launcher-3606-thread-1) [n:127.0.0.1:34679_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_19F448DC1133CDE-001/tempDir-001/node2/.
[junit4] 2> 3394740 INFO (jetty-launcher-3606-thread-2) [n:127.0.0.1:34068_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34068.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@74421fac
[junit4] 2> 3394746 INFO (jetty-launcher-3606-thread-2) [n:127.0.0.1:34068_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34068.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@74421fac
[junit4] 2> 3394746 INFO (jetty-launcher-3606-thread-2) [n:127.0.0.1:34068_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34068.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@74421fac
[junit4] 2> 3394747 INFO (jetty-launcher-3606-thread-2) [n:127.0.0.1:34068_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_19F448DC1133CDE-001/tempDir-001/node1/.
[junit4] 2> 3394788 INFO (SUITE-HdfsRecoveryZkTest-seed#[19F448DC1133CDE]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 3394790 INFO (SUITE-HdfsRecoveryZkTest-seed#[19F448DC1133CDE]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:39024/solr ready
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 3394848 WARN (SUITE-HdfsRecoveryZkTest-seed#[19F448DC1133CDE]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 3394872 WARN (SUITE-HdfsRecoveryZkTest-seed#[19F448DC1133CDE]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 3394874 INFO (SUITE-HdfsRecoveryZkTest-seed#[19F448DC1133CDE]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 3394903 INFO (SUITE-HdfsRecoveryZkTest-seed#[19F448DC1133CDE]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_57247_hdfs____.dxx14z/webapp
[junit4] 2> 3395436 INFO (SUITE-HdfsRecoveryZkTest-seed#[19F448DC1133CDE]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:57247
[junit4] 2> 3395630 WARN (SUITE-HdfsRecoveryZkTest-seed#[19F448DC1133CDE]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 3395632 INFO (SUITE-HdfsRecoveryZkTest-seed#[19F448DC1133CDE]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 3395674 INFO (SUITE-HdfsRecoveryZkTest-seed#[19F448DC1133CDE]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_42684_datanode____.bhdbj9/webapp
[junit4] 2> 3396123 INFO (SUITE-HdfsRecoveryZkTest-seed#[19F448DC1133CDE]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42684
[junit4] 2> 3396255 WARN (SUITE-HdfsRecoveryZkTest-seed#[19F448DC1133CDE]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 3396256 INFO (SUITE-HdfsRecoveryZkTest-seed#[19F448DC1133CDE]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 3396288 INFO (SUITE-HdfsRecoveryZkTest-seed#[19F448DC1133CDE]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_47580_datanode____bt4ne9/webapp
[junit4] 2> 3396374 INFO (IPC Server handler 4 on 52272) [ ] BlockStateChange BLOCK* processReport: from storage DS-850c8b69-51e8-42d5-934d-63fadfd5ac8a node DatanodeRegistration(127.0.0.1:43451, datanodeUuid=2715034c-acfe-4a52-9fcc-cecf0bfbc8d9, infoPort=34779, infoSecurePort=0, ipcPort=56637, storageInfo=lv=-56;cid=testClusterID;nsid=1651850002;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 3396374 INFO (IPC Server handler 4 on 52272) [ ] BlockStateChange BLOCK* processReport: from storage DS-c913d72d-e2f6-4028-8911-ae86baca4f96 node DatanodeRegistration(127.0.0.1:43451, datanodeUuid=2715034c-acfe-4a52-9fcc-cecf0bfbc8d9, infoPort=34779, infoSecurePort=0, ipcPort=56637, storageInfo=lv=-56;cid=testClusterID;nsid=1651850002;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 3396818 INFO (SUITE-HdfsRecoveryZkTest-seed#[19F448DC1133CDE]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:47580
[junit4] 2> 3397073 INFO (IPC Server handler 7 on 52272) [ ] BlockStateChange BLOCK* processReport: from storage DS-457882e2-9a3c-4f09-9d8f-fe49dc41e407 node DatanodeRegistration(127.0.0.1:60661, datanodeUuid=e515b5aa-3e86-4682-a357-9fd352bff9d7, infoPort=40849, infoSecurePort=0, ipcPort=46032, storageInfo=lv=-56;cid=testClusterID;nsid=1651850002;c=0), blocks: 0, hasStaleStorage: true, processing time: 1 msecs
[junit4] 2> 3397073 INFO (IPC Server handler 7 on 52272) [ ] BlockStateChange BLOCK* processReport: from storage DS-691c7a32-7c0b-4b27-8670-e0a995751ca5 node DatanodeRegistration(127.0.0.1:60661, datanodeUuid=e515b5aa-3e86-4682-a357-9fd352bff9d7, infoPort=40849, infoSecurePort=0, ipcPort=46032, storageInfo=lv=-56;cid=testClusterID;nsid=1651850002;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 3397454 INFO (TEST-HdfsRecoveryZkTest.test-seed#[19F448DC1133CDE]) [ ] o.a.s.SolrTestCaseJ4 ###Starting test
[junit4] 2> 3397456 INFO (qtp1324117838-32435) [n:127.0.0.1:34068_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf&maxShardsPerNode=1&name=recoverytest&nrtReplicas=2&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 3397461 INFO (OverseerThreadFactory-8151-thread-1-processing-n:127.0.0.1:34679_solr) [n:127.0.0.1:34679_solr ] o.a.s.c.CreateCollectionCmd Create collection recoverytest
[junit4] 2> 3397581 INFO (qtp1324117838-32439) [n:127.0.0.1:34068_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=recoverytest_shard1_replica_n1&action=CREATE&numShards=1&collection=recoverytest&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 3397582 INFO (qtp1324117838-32439) [n:127.0.0.1:34068_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 3397583 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=recoverytest_shard1_replica_n2&action=CREATE&numShards=1&collection=recoverytest&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 3397583 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 3397687 INFO (zkCallback-3617-thread-1-processing-n:127.0.0.1:34068_solr) [n:127.0.0.1:34068_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/recoverytest/state.json] for collection [recoverytest] has occurred - updating... (live nodes size: [2])
[junit4] 2> 3397688 INFO (zkCallback-3618-thread-2-processing-n:127.0.0.1:34679_solr) [n:127.0.0.1:34679_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/recoverytest/state.json] for collection [recoverytest] has occurred - updating... (live nodes size: [2])
[junit4] 2> 3398630 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 3398630 INFO (qtp1324117838-32439) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 3398644 INFO (qtp1324117838-32439) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n1] o.a.s.s.IndexSchema [recoverytest_shard1_replica_n1] Schema name=minimal
[junit4] 2> 3398647 INFO (qtp1324117838-32439) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 3398647 INFO (qtp1324117838-32439) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'recoverytest_shard1_replica_n1' using configuration from collection recoverytest, trusted=true
[junit4] 2> 3398648 INFO (qtp1324117838-32439) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34068.solr.core.recoverytest.shard1.replica_n1' (registry 'solr.core.recoverytest.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@74421fac
[junit4] 2> 3398648 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n2] o.a.s.s.IndexSchema [recoverytest_shard1_replica_n2] Schema name=minimal
[junit4] 2> 3398666 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 3398666 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'recoverytest_shard1_replica_n2' using configuration from collection recoverytest, trusted=true
[junit4] 2> 3398666 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34679.solr.core.recoverytest.shard1.replica_n2' (registry 'solr.core.recoverytest.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@74421fac
[junit4] 2> 3398667 INFO (qtp1324117838-32439) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:52272/data
[junit4] 2> 3398667 INFO (qtp1324117838-32439) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 3398668 INFO (qtp1324117838-32439) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 3398668 INFO (qtp1324117838-32439) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n1] o.a.s.c.SolrCore [[recoverytest_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_19F448DC1133CDE-001/tempDir-001/node1/recoverytest_shard1_replica_n1], dataDir=[hdfs://localhost:52272/data/recoverytest/core_node1/data/]
[junit4] 2> 3398670 INFO (qtp1324117838-32439) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:52272/data/recoverytest/core_node1/data/snapshot_metadata
[junit4] 2> 3398670 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n2] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:52272/data
[junit4] 2> 3398670 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n2] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 3398670 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 3398670 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n2] o.a.s.c.SolrCore [[recoverytest_shard1_replica_n2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_19F448DC1133CDE-001/tempDir-001/node2/recoverytest_shard1_replica_n2], dataDir=[hdfs://localhost:52272/data/recoverytest/core_node2/data/]
[junit4] 2> 3398681 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:52272/data/recoverytest/core_node2/data/snapshot_metadata
[junit4] 2> 3398684 INFO (qtp1324117838-32439) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 3398684 INFO (qtp1324117838-32439) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 3398684 INFO (qtp1324117838-32439) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 3398725 INFO (qtp1324117838-32439) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 3398726 INFO (qtp1324117838-32439) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:52272/data/recoverytest/core_node1/data
[junit4] 2> 3398727 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n2] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 3398727 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n2] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 3398727 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n2] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 3398743 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n2] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 3398744 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:52272/data/recoverytest/core_node2/data
[junit4] 2> 3398772 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:52272/data/recoverytest/core_node2/data/index
[junit4] 2> 3398779 INFO (qtp1324117838-32439) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:52272/data/recoverytest/core_node1/data/index
[junit4] 2> 3398788 INFO (qtp1324117838-32439) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 3398788 INFO (qtp1324117838-32439) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 3398788 INFO (qtp1324117838-32439) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 3398790 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n2] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 3398790 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n2] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
[junit4] 2> 3398790 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n2] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 3398815 INFO (qtp1324117838-32439) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 3398819 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n2] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 3398908 INFO (IPC Server handler 9 on 52272) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:43451 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-457882e2-9a3c-4f09-9d8f-fe49dc41e407:NORMAL:127.0.0.1:60661|RBW], ReplicaUC[[DISK]DS-c913d72d-e2f6-4028-8911-ae86baca4f96:NORMAL:127.0.0.1:43451|FINALIZED]]} size 0
[junit4] 2> 3398909 INFO (IPC Server handler 8 on 52272) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:43451 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-457882e2-9a3c-4f09-9d8f-fe49dc41e407:NORMAL:127.0.0.1:60661|RBW], ReplicaUC[[DISK]DS-850c8b69-51e8-42d5-934d-63fadfd5ac8a:NORMAL:127.0.0.1:43451|RBW]]} size 0
[junit4] 2> 3398912 INFO (IPC Server handler 6 on 52272) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:60661 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-c913d72d-e2f6-4028-8911-ae86baca4f96:NORMAL:127.0.0.1:43451|FINALIZED], ReplicaUC[[DISK]DS-691c7a32-7c0b-4b27-8670-e0a995751ca5:NORMAL:127.0.0.1:60661|FINALIZED]]} size 0
[junit4] 2> 3398913 INFO (IPC Server handler 2 on 52272) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:60661 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-457882e2-9a3c-4f09-9d8f-fe49dc41e407:NORMAL:127.0.0.1:60661|RBW], ReplicaUC[[DISK]DS-850c8b69-51e8-42d5-934d-63fadfd5ac8a:NORMAL:127.0.0.1:43451|RBW]]} size 0
[junit4] 2> 3399079 INFO (qtp1324117838-32439) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 3399079 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 3399079 INFO (qtp1324117838-32439) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 3399079 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 3399079 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n2] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
[junit4] 2> 3399079 INFO (qtp1324117838-32439) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
[junit4] 2> 3399101 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 3399101 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 3399108 INFO (qtp1324117838-32439) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 3399108 INFO (qtp1324117838-32439) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 3399127 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@56a8a76d[recoverytest_shard1_replica_n2] main]
[junit4] 2> 3399130 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 3399131 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 3399131 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 3399134 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1569602833095852032
[junit4] 2> 3399136 INFO (searcherExecutor-8157-thread-1-processing-n:127.0.0.1:34679_solr x:recoverytest_shard1_replica_n2 s:shard1 c:recoverytest) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n2] o.a.s.c.SolrCore [recoverytest_shard1_replica_n2] Registered new searcher Searcher@56a8a76d[recoverytest_shard1_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 3399139 INFO (qtp1324117838-32439) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@39a409b9[recoverytest_shard1_replica_n1] main]
[junit4] 2> 3399142 INFO (qtp1324117838-32439) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 3399143 INFO (qtp1324117838-32439) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 3399144 INFO (qtp1324117838-32439) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 3399145 INFO (searcherExecutor-8156-thread-1-processing-n:127.0.0.1:34068_solr x:recoverytest_shard1_replica_n1 s:shard1 c:recoverytest) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n1] o.a.s.c.SolrCore [recoverytest_shard1_replica_n1] Registered new searcher Searcher@39a409b9[recoverytest_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 3399147 INFO (qtp1324117838-32439) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1569602833109483520
[junit4] 2> 3399150 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9999ms
[junit4] 2> 3399255 INFO (zkCallback-3617-thread-1-processing-n:127.0.0.1:34068_solr) [n:127.0.0.1:34068_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/recoverytest/state.json] for collection [recoverytest] has occurred - updating... (live nodes size: [2])
[junit4] 2> 3399255 INFO (zkCallback-3618-thread-2-processing-n:127.0.0.1:34679_solr) [n:127.0.0.1:34679_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/recoverytest/state.json] for collection [recoverytest] has occurred - updating... (live nodes size: [2])
[junit4] 2> 3399651 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 3399651 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 3399651 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/
[junit4] 2> 3399652 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n2] o.a.s.u.PeerSync PeerSync: core=recoverytest_shard1_replica_n2 url=http://127.0.0.1:34679/solr START replicas=[http://127.0.0.1:34068/solr/recoverytest_shard1_replica_n1/] nUpdates=100
[junit4] 2> 3399652 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n2] o.a.s.u.PeerSync PeerSync: core=recoverytest_shard1_replica_n2 url=http://127.0.0.1:34679/solr DONE. We have no versions. sync failed.
[junit4] 2> 3399655 INFO (qtp1324117838-32441) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.c.S.Request [recoverytest_shard1_replica_n1] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 3399655 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n2] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 3399655 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 3399656 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 3399660 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/ shard1
[junit4] 2> 3399765 INFO (zkCallback-3617-thread-1-processing-n:127.0.0.1:34068_solr) [n:127.0.0.1:34068_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/recoverytest/state.json] for collection [recoverytest] has occurred - updating... (live nodes size: [2])
[junit4] 2> 3399765 INFO (zkCallback-3618-thread-2-processing-n:127.0.0.1:34679_solr) [n:127.0.0.1:34679_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/recoverytest/state.json] for collection [recoverytest] has occurred - updating... (live nodes size: [2])
[junit4] 2> 3399811 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 3399814 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=recoverytest_shard1_replica_n2&action=CREATE&numShards=1&collection=recoverytest&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2230
[junit4] 2> 3399917 INFO (zkCallback-3617-thread-1-processing-n:127.0.0.1:34068_solr) [n:127.0.0.1:34068_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/recoverytest/state.json] for collection [recoverytest] has occurred - updating... (live nodes size: [2])
[junit4] 2> 3399917 INFO (zkCallback-3618-thread-2-processing-n:127.0.0.1:34679_solr) [n:127.0.0.1:34679_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/recoverytest/state.json] for collection [recoverytest] has occurred - updating... (live nodes size: [2])
[junit4] 2> 3400159 INFO (qtp1324117838-32439) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 x:recoverytest_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=recoverytest_shard1_replica_n1&action=CREATE&numShards=1&collection=recoverytest&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2577
[junit4] 2> 3400166 INFO (qtp1324117838-32435) [n:127.0.0.1:34068_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 3400268 INFO (zkCallback-3618-thread-2-processing-n:127.0.0.1:34679_solr) [n:127.0.0.1:34679_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/recoverytest/state.json] for collection [recoverytest] has occurred - updating... (live nodes size: [2])
[junit4] 2> 3400268 INFO (zkCallback-3617-thread-1-processing-n:127.0.0.1:34068_solr) [n:127.0.0.1:34068_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/recoverytest/state.json] for collection [recoverytest] has occurred - updating... (live nodes size: [2])
[junit4] 2> 3401166 INFO (qtp1324117838-32435) [n:127.0.0.1:34068_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=2&collection.configName=conf&maxShardsPerNode=1&name=recoverytest&nrtReplicas=2&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=3710
[junit4] 2> 3401168 INFO (TEST-HdfsRecoveryZkTest.test-seed#[19F448DC1133CDE]) [ ] o.a.s.c.RecoveryZkTest Indexing 30000 documents
[junit4] 2> 3401211 DEBUG (qtp837932411-32438) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.HdfsTransactionLog Opening new tlog hdfs tlog{file=hdfs://localhost:52272/data/recoverytest/core_node2/data/tlog/tlog.0000000000000000000 refcount=1}
[junit4] 2> 3401246 DEBUG (qtp1324117838-32443) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.HdfsTransactionLog Opening new tlog hdfs tlog{file=hdfs://localhost:52272/data/recoverytest/core_node1/data/tlog/tlog.0000000000000000000 refcount=1}
[junit4] 2> 3401263 INFO (qtp1324117838-32429) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{add=[2-0 (1569602835244384256)]} 0 30
[junit4] 2> 3401263 INFO (qtp837932411-32442) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-0 (1569602835244384256)]} 0 80
[junit4] 2> 3401264 INFO (qtp1324117838-32443) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{add=[1-0 (1569602835241238528)]} 0 31
[junit4] 2> 3401264 INFO (qtp837932411-32438) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-0 (1569602835241238528)]} 0 83
[junit4] 2> 3401266 INFO (qtp1324117838-32441) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{add=[2-1 (1569602835329318912)]} 0 0
[junit4] 2> 3401266 INFO (qtp837932411-32428) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-1 (1569602835329318912)]} 0 2
[junit4] 2> 3401267 INFO (qtp1324117838-32439) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{add=[1-1 (1569602835330367488)]} 0 1
[junit4] 2> 3401269 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-1 (1569602835330367488)]} 0 4
[junit4] 2> 3401276 INFO (qtp1324117838-32435) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{delete=[2-1 (-1569602835333513216)]} 0 5
[junit4] 2> 3401277 INFO (qtp837932411-32430) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{delete=[2-1 (-1569602835333513216)]} 0 9
[junit4] 2> 3401283 INFO (qtp1324117838-32431) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{add=[2-2 (1569602835342950400)]} 0 5
[junit4] 2> 3401284 INFO (qtp837932411-32440) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-2 (1569602835342950400)]} 0 6
[junit4] 2> 3401287 INFO (qtp1324117838-32429) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{delete=[1-1 (-1569602835345047552)]} 0 7
[junit4] 2> 3401287 INFO (qtp837932411-32442) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{delete=[1-1 (-1569602835345047552)]} 0 8
[junit4] 2> 3401289 INFO (qtp1324117838-32443) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{add=[2-3 (1569602835351339008)]} 0 3
[junit4] 2> 3401290 INFO (qtp837932411-32438) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-3 (1569602835351339008)]} 0 4
[junit4] 2> 3401291 INFO (qtp1324117838-32441) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{add=[1-2 (1569602835354484736)]} 0 2
[junit4] 2> 3401291 INFO (qtp837932411-32428) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-2 (1569602835354484736)]} 0 3
[junit4] 2> 3401295 INFO (qtp1324117838-32439) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{add=[2-4 (1569602835357630464)]} 0 1
[junit4] 2> 3401295 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-4 (1569602835357630464)]} 0 3
[junit4] 2> 3401295 INFO (qtp1324117838-32435) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{add=[1-3 (1569602835360776192)]} 0 0
[junit4] 2> 3401295 INFO (qtp837932411-32430) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-3 (1569602835360776192)]} 0 1
[junit4] 2> 3401297 INFO (qtp1324117838-32439) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{add=[2-5 (1569602835361824768)]} 0 0
[junit4] 2> 3401297 INFO (qtp837932411-32440) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-5 (1569602835361824768)]} 0 1
[junit4] 2> 3401297 INFO (qtp1324117838-32429) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{add=[1-4 (1569602835362873344)]} 0 0
[junit4] 2> 3401297 INFO (qtp837932411-32442) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-4 (1569602835362873344)]} 0 1
[junit4] 2> 3401299 INFO (qtp1324117838-32443) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{add=[2-6 (1569602835363921920)]} 0 1
[junit4] 2> 3401300 INFO (qtp837932411-32438) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-6 (1569602835363921920)]} 0 2
[junit4] 2> 3401301 INFO (qtp1324117838-32441) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{add=[1-5 (1569602835366019072)]} 0 1
[junit4] 2> 3401301 INFO (qtp837932411-32428) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-5 (1569602835366019072)]} 0 2
[junit4] 2> 3401304 INFO (qtp1324117838-32443) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{delete=[2-6 (-1569602835368116224)]} 0 1
[junit4] 2> 3401304 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{delete=[2-6 (-1569602835368116224)]} 0 3
[junit4] 2> 3401304 INFO (qtp1324117838-32435) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{add=[1-6 (1569602835369164800)]} 0 1
[junit4] 2> 3401305 INFO (qtp837932411-32430) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-6 (1569602835369164800)]} 0 3
[junit4] 2> 3401309 INFO (qtp1324117838-32439) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{delete=[1-6 (-1569602835372310529)]} 0 2
[junit4] 2> 3401310 INFO (qtp837932411-32442) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{delete=[1-6 (-1569602835372310529)]} 0 4
[junit4] 2> 3401310 INFO (qtp1324117838-32429) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{add=[2-7 (1569602835372310528)]} 0 3
[junit4] 2> 3401311 INFO (qtp837932411-32440) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-7 (1569602835372310528)]} 0 5
[junit4] 2> 3401313 INFO (qtp1324117838-32431) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{add=[1-7 (1569602835377553408)]} 0 1
[junit4] 2> 3401313 INFO (qtp837932411-32438) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-7 (1569602835377553408)]} 0 2
[junit4] 2> 3401314 INFO (qtp1324117838-32441) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{add=[2-8 (1569602835378601984)]} 0 1
[junit4] 2> 3401314 INFO (qtp837932411-32428) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-8 (1569602835378601984)]} 0 2
[junit4] 2> 3401315 INFO (qtp1324117838-32443) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{add=[1-8 (1569602835380699136)]} 0 1
[junit4] 2> 3401315 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-8 (1569602835380699136)]} 0 2
[junit4] 2> 3401315 INFO (qtp1324117838-32435) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{add=[2-9 (1569602835381747712)]} 0 0
[junit4] 2> 3401316 INFO (qtp837932411-32430) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-9 (1569602835381747712)]} 0 1
[junit4] 2> 3401317 INFO (qtp1324117838-32439) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{add=[1-9 (1569602835383844864)]} 0 1
[junit4] 2> 3401318 INFO (qtp837932411-32442) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-9 (1569602835383844864)]} 0 1
[junit4] 2> 3401318 INFO (qtp1324117838-32429) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{add=[2-10 (1569602835383844865)]} 0 0
[junit4] 2> 3401318 INFO (qtp837932411-32440) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-10 (1569602835383844865)]} 0 1
[junit4] 2> 3401321 INFO (qtp1324117838-32431) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{add=[1-10 (1569602835385942016)]} 0 2
[junit4] 2> 3401322 INFO (qtp837932411-32438) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-10 (1569602835385942016)]} 0 3
[junit4] 2> 3401322 INFO (qtp1324117838-32441) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{delete=[2-8 (-1569602835386990592)]} 0 1
[junit4] 2> 3401323 INFO (qtp837932411-32440) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{delete=[2-8 (-1569602835386990592)]} 0 4
[junit4] 2> 3401324 INFO (qtp1324117838-32443) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{delete=[1-8 (-1569602835391184896)]} 0 0
[junit4] 2> 3401324 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{delete=[1-8 (-1569602835391184896)]} 0 1
[junit4] 2> 3401325 INFO (qtp1324117838-32435) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{add=[2-11 (1569602835391184897)]} 0 0
[junit4] 2> 3401325 INFO (qtp837932411-32430) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-11 (1569602835391184897)]} 0 1
[junit4] 2> 3401326 INFO (qtp1324117838-32439) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{add=[1-11 (1569602835393282048)]} 0 1
[junit4] 2> 3401326 INFO (qtp837932411-32442) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-11 (1569602835393282048)]} 0 1
[junit4] 2> 3401326 INFO (qtp1324117838-32429) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{delete=[2-10 (-1569602835393282049)]} 0 0
[junit4] 2> 3401327 INFO (qtp837932411-32428) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{delete=[2-10 (-1569602835393282049)]} 0 1
[junit4] 2> 3401328 INFO (qtp1324117838-32431) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{delete=[1-10 (-1569602835395379200)]} 0 0
[junit4] 2> 3401328 INFO (qtp837932411-32438) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{delete=[1-10 (-1569602835395379200)]} 0 1
[junit4] 2> 3401329 INFO (qtp1324117838-32441) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{add=[2-12 (1569602835395379201)]} 0 1
[junit4] 2> 3401329 INFO (qtp837932411-32440) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-12 (1569602835395379201)]} 0 1
[junit4] 2> 3401331 INFO (qtp1324117838-32443) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{add=[1-12 (1569602835397476352)]} 0 1
[junit4] 2> 3401331 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-12 (1569602835397476352)]} 0 1
[junit4] 2> 3401331 INFO (qtp1324117838-32435) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{add=[2-13 (1569602835398524928)]} 0 0
[junit4] 2> 3401331 INFO (qtp837932411-32430) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-13 (1569602835398524928)]} 0 1
[junit4] 2> 3401335 INFO (qtp1324117838-32439) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{add=[1-13 (1569602835399573504)]} 0 2
[junit4] 2> 3401335 INFO (qtp837932411-32442) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-13 (1569602835399573504)]} 0 3
[junit4] 2> 3401335 INFO (qtp1324117838-32429) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{delete=[2-11 (-1569602835400622080)]} 0 2
[junit4] 2> 3401335 INFO (qtp837932411-32428) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{delete=[2-11 (-1569602835400622080)]} 0 3
[junit4] 2> 3401336 INFO (qtp1324117838-32431) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{delete=[1-11 (-1569602835403767808)]} 0 0
[junit4] 2> 3401337 INFO (qtp837932411-32438) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{delete=[1-11 (-1569602835403767808)]} 0 1
[junit4] 2> 3401337 INFO (qtp1324117838-32441) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{add=[2-14 (1569602835404816384)]} 0 0
[junit4] 2> 3401338 INFO (qtp837932411-32440) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-14 (1569602835404816384)]} 0 1
[junit4] 2> 3401342 INFO (qtp1324117838-32443) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{add=[1-14 (1569602835405864960)]} 0 4
[junit4] 2> 3401343 INFO (qtp1324117838-32435) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{delete=[2-12 (-1569602835406913536)]} 0 4
[junit4] 2> 3401343 INFO (qtp837932411-32430) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{delete=[2-12 (-1569602835406913536)]} 0 5
[junit4] 2> 3401345 INFO (qtp837932411-32434) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-14 (1569602835405864960)]} 0 7
[junit4] 2> 3401346 INFO (qtp1324117838-32439) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{add=[2-15 (1569602835414253568)]} 0 0
[junit4] 2> 3401347 INFO (qtp837932411-32442) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-15 (1569602835414253568)]} 0 1
[junit4] 2> 3401348 INFO (qtp1324117838-32429) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{delete=[1-12 (-1569602835415302144)]} 0 1
[junit4] 2> 3401348 INFO (qtp837932411-32428) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{delete=[1-12 (-1569602835415302144)]} 0 2
[junit4] 2> 3401348 INFO (qtp1324117838-32439) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{add=[2-16 (1569602835416350720)]} 0 0
[junit4] 2> 3401349 INFO (qtp837932411-32438) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{add=[2-16 (1569602835416350720)]} 0 1
[junit4] 2> 3401350 INFO (qtp1324117838-32441) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{add=[1-15 (1569602835418447872)]} 0 0
[junit4] 2> 3401351 INFO (qtp837932411-32440) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=2}{add=[1-15 (1569602835418447872)]} 0 2
[junit4] 2> 3401353 INFO (qtp1324117838-32443) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{delete=[2-13 (-1569602835418447873)]} 0 3
[junit4] 2> 3401353 INFO (qtp1324117838-32435) [n:127.0.0.1:34068_solr c:recoverytest s:shard1 r:core_node1 x:recoverytest_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:34679/solr/recoverytest_shard1_replica_n2/&wt=javabin&version=2}{add=[1-16 (1569602835420545024)]} 0 1
[junit4] 2> 3401353 INFO (qtp837932411-32438) [n:127.0.0.1:34679_solr c:recoverytest s:shard1 r:core_node2 x:recoverytest_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [recoverytest_shard1_replica_n2] webapp=/solr path=/update params={wt=javabin&version=
[...truncated too long message...]
at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1705)
[junit4] 2> at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:204)
[junit4] 2> at org.apache.solr.cloud.hdfs.HdfsRecoveryZkTest.teardownClass(HdfsRecoveryZkTest.java:53)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> Caused by: java.lang.NullPointerException
[junit4] 2> at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:203)
[junit4] 2> at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3370)
[junit4] 2> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:5729)
[junit4] 2> ... 54 more
[junit4] 2> 3419795 INFO (SUITE-HdfsRecoveryZkTest-seed#[19F448DC1133CDE]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:39024 39024
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_19F448DC1133CDE-001
[junit4] 2> Jun 08, 2017 2:45:29 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 35 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {rnd_b=PostingsFormat(name=Direct), _version_=PostingsFormat(name=Asserting), a_t=FSTOrd50, a_i=PostingsFormat(name=Direct), id=PostingsFormat(name=Direct)}, docValues:{}, maxPointsInLeafNode=38, maxMBSortInHeap=5.620767783290915, sim=RandomSimilarity(queryNorm=false): {}, locale=sl-SI, timezone=Europe/Astrakhan
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_121 (64-bit)/cpus=4,threads=2,free=183574208,total=522715136
[junit4] 2> NOTE: All tests run in this JVM: [TestSystemIdResolver, CdcrUpdateLogTest, MoreLikeThisHandlerTest, ClusterStateUpdateTest, JvmMetricsTest, MigrateRouteKeyTest, DirectSolrConnectionTest, TestFieldCache, BigEndianAscendingWordDeserializerTest, DocumentBuilderTest, TestQueryTypes, TestDistribDocBasedVersion, TestFiltering, TestDelegationWithHadoopAuth, OverseerRolesTest, OverriddenZkACLAndCredentialsProvidersTest, DistributedFacetPivotLargeTest, TestLegacyTerms, PreAnalyzedUpdateProcessorTest, ZkControllerTest, TestLegacyField, DistanceFunctionTest, TestAddFieldRealTimeGet, SolrMetricManagerTest, TestFieldCacheSortRandom, ReplaceNodeTest, TestUseDocValuesAsStored, DistanceUnitsTest, TestOnReconnectListenerSupport, DocExpirationUpdateProcessorFactoryTest, TestSweetSpotSimilarityFactory, TestJavabinTupleStreamParser, SOLR749Test, TestReRankQParserPlugin, HttpSolrCallGetCoreTest, TestExportWriter, TestBulkSchemaAPI, SpellCheckCollatorTest, TestRawTransformer, VMParamsZkACLAndCredentialsProvidersTest, TestSolrJ, TestOverriddenPrefixQueryForCustomFieldType, WordBreakSolrSpellCheckerTest, BadCopyFieldTest, AssignTest, BasicAuthStandaloneTest, TestSSLRandomization, TestRandomFlRTGCloud, DistributedIntervalFacetingTest, TestStressVersions, TestHdfsCloudBackupRestore, SchemaVersionSpecificBehaviorTest, IndexSchemaTest, TestUninvertingReader, TestCloudSchemaless, TestNonDefinedSimilarityFactory, SimpleMLTQParserTest, SpellCheckComponentTest, TestComplexPhraseQParserPlugin, UpdateRequestProcessorFactoryTest, QueryParsingTest, InfixSuggestersTest, SampleTest, AlternateDirectoryTest, ZkCLITest, CollectionsAPIDistributedZkTest, MergeStrategyTest, CdcrBootstrapTest, BlockCacheTest, SolrGangliaReporterTest, ResourceLoaderTest, UninvertDocValuesMergePolicyTest, TestInfoStreamLogging, TestCharFilters, SolrSlf4jReporterTest, TermVectorComponentTest, TestSchemaVersionResource, ChaosMonkeySafeLeaderWithPullReplicasTest, DistributedVersionInfoTest, HttpPartitionTest, DistributedDebugComponentTest, TestConfigSetProperties, CSVRequestHandlerTest, DistribDocExpirationUpdateProcessorTest, DateRangeFieldTest, TestElisionMultitermQuery, TermsComponentTest, BadComponentTest, TestCursorMarkWithoutUniqueKey, TestSortByMinMaxFunction, ChaosMonkeySafeLeaderTest, TestLegacyFieldCache, TestExactStatsCache, TestSolrCoreSnapshots, TestCloudNestedDocsSort, TestSolrCloudWithSecureImpersonation, TestJsonFacets, TestHdfsUpdateLog, TestFilteredDocIdSet, TestPostingsSolrHighlighter, TestManagedResource, DistributedTermsComponentTest, TestFieldResource, TestTestInjection, TestReplicaProperties, ConvertedLegacyTest, TestWordDelimiterFilterFactory, CdcrRequestHandlerTest, TestNumericRangeQuery64, SolrIndexSplitterTest, TestSimpleTrackingShardHandler, SystemInfoHandlerTest, SimplePostToolTest, TestPartialUpdateDeduplication, CreateCollectionCleanupTest, TestUpdate, UpdateLogTest, AtomicUpdateProcessorFactoryTest, TestSolr4Spatial2, IndexBasedSpellCheckerTest, MetricsHandlerTest, BasicDistributedZkTest, UnloadDistributedZkTest, OpenCloseCoreStressTest, TestRandomFaceting, ZkSolrClientTest, TestZkChroot, ShardRoutingCustomTest, TestDistributedGrouping, TestFaceting, TestHashPartitioner, DistributedSpellCheckComponentTest, TestRealTimeGet, TestStressReorder, TestJoin, SolrCmdDistributorTest, BadIndexSchemaTest, TestSort, BasicFunctionalityTest, TestIndexSearcher, HighlighterTest, SoftAutoCommitTest, XsltUpdateRequestHandlerTest, CacheHeaderTest, TestQueryUtils, TestWriterPerf, TestOmitPositions, TestValueSourceCache, FieldAnalysisRequestHandlerTest, IndexSchemaRuntimeFieldTest, RegexBoostProcessorTest, ReturnFieldsTest, TestCSVResponseWriter, UniqFieldsUpdateProcessorFactoryTest, PingRequestHandlerTest, HighlighterConfigTest, TestSolrIndexConfig, TestQuerySenderNoQuery, ResponseLogComponentTest, TestStressRecovery, TestEmbeddedSolrServerConstructors, TestEmbeddedSolrServerSchemaAPI, ConnectionReuseTest, CdcrVersionReplicationTest, CleanupOldIndexTest, CloudExitableDirectoryReaderTest, CollectionStateFormat2Test, CollectionTooManyReplicasTest, CollectionsAPIAsyncDistributedZkTest, DeleteReplicaTest, DistribCursorPagingTest, LeaderInitiatedRecoveryOnShardRestartTest, MissingSegmentRecoveryTest, MoveReplicaTest, OverseerCollectionConfigSetProcessorTest, PeerSyncReplicationTest, RecoveryAfterSoftCommitTest, RollingRestartTest, SharedFSAutoReplicaFailoverUtilsTest, SimpleCollectionCreateDeleteTest, SolrCloudExampleTest, SolrXmlInZkTest, TestAuthenticationFramework, TestCloudInspectUtil, TestCloudPseudoReturnFields, TestCloudRecovery, TestClusterProperties, TestCollectionAPI, TestLeaderElectionWithEmptyReplica, TestMiniSolrCloudCluster, TestSegmentSorting, TestSizeLimitedDistributedMap, TestSolrCloudWithDelegationTokens, TestSolrCloudWithKerberosAlt, TestStressCloudBlindAtomicUpdates, HdfsBasicDistributedZk2Test, HdfsNNFailoverTest, HdfsRecoverLeaseTest, HdfsRecoveryZkTest]
[junit4] Completed [568/726 (1!)] on J1 in 35.20s, 1 test, 1 failure <<< FAILURES!
[...truncated 7964 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/build.xml:783: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/build.xml:727: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid31990.hprof
Total time: 269 minutes 10 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1317 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1317/
1 tests failed.
FAILED: org.apache.solr.cloud.hdfs.HdfsBasicDistributedZk2Test.test
Error Message:
.response.numFound:36!=67
Stack Trace:
junit.framework.AssertionFailedError: .response.numFound:36!=67
at __randomizedtesting.SeedInfo.seed([5F9BD9DF2B6C06E8:D7CFE60585906B10]:0)
at junit.framework.Assert.fail(Assert.java:50)
at org.apache.solr.BaseDistributedSearchTestCase.compareSolrResponses(BaseDistributedSearchTestCase.java:913)
at org.apache.solr.BaseDistributedSearchTestCase.compareResponses(BaseDistributedSearchTestCase.java:932)
at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:607)
at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:587)
at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:566)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkQueries(AbstractFullDistribZkTestBase.java:923)
at org.apache.solr.cloud.BasicDistributedZk2Test.test(BasicDistributedZk2Test.java:98)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 11438 lines...]
[junit4] Suite: org.apache.solr.cloud.hdfs.HdfsBasicDistributedZk2Test
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_5F9BD9DF2B6C06E8-001/init-core-data-001
[junit4] 2> 9994 WARN (SUITE-HdfsBasicDistributedZk2Test-seed#[5F9BD9DF2B6C06E8]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=4 numCloses=4
[junit4] 2> 9994 INFO (SUITE-HdfsBasicDistributedZk2Test-seed#[5F9BD9DF2B6C06E8]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields
[junit4] 2> 9996 INFO (SUITE-HdfsBasicDistributedZk2Test-seed#[5F9BD9DF2B6C06E8]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
[junit4] 2> 9996 INFO (SUITE-HdfsBasicDistributedZk2Test-seed#[5F9BD9DF2B6C06E8]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 2> 10872 WARN (SUITE-HdfsBasicDistributedZk2Test-seed#[5F9BD9DF2B6C06E8]-worker) [ ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 12266 WARN (SUITE-HdfsBasicDistributedZk2Test-seed#[5F9BD9DF2B6C06E8]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 12794 INFO (SUITE-HdfsBasicDistributedZk2Test-seed#[5F9BD9DF2B6C06E8]-worker) [ ] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
[junit4] 2> 12838 WARN (SUITE-HdfsBasicDistributedZk2Test-seed#[5F9BD9DF2B6C06E8]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 13014 INFO (SUITE-HdfsBasicDistributedZk2Test-seed#[5F9BD9DF2B6C06E8]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 13150 INFO (SUITE-HdfsBasicDistributedZk2Test-seed#[5F9BD9DF2B6C06E8]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_38434_hdfs____vdx67u/webapp
[junit4] 2> 14888 INFO (SUITE-HdfsBasicDistributedZk2Test-seed#[5F9BD9DF2B6C06E8]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:38434
[junit4] 2> 17106 WARN (SUITE-HdfsBasicDistributedZk2Test-seed#[5F9BD9DF2B6C06E8]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 17112 INFO (SUITE-HdfsBasicDistributedZk2Test-seed#[5F9BD9DF2B6C06E8]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 17128 INFO (SUITE-HdfsBasicDistributedZk2Test-seed#[5F9BD9DF2B6C06E8]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_43866_datanode____.y2paps/webapp
[junit4] 2> 17865 INFO (SUITE-HdfsBasicDistributedZk2Test-seed#[5F9BD9DF2B6C06E8]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43866
[junit4] 2> 18517 WARN (SUITE-HdfsBasicDistributedZk2Test-seed#[5F9BD9DF2B6C06E8]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 18519 INFO (SUITE-HdfsBasicDistributedZk2Test-seed#[5F9BD9DF2B6C06E8]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 18544 INFO (SUITE-HdfsBasicDistributedZk2Test-seed#[5F9BD9DF2B6C06E8]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_52175_datanode____vt3t2x/webapp
[junit4] 2> 19473 INFO (SUITE-HdfsBasicDistributedZk2Test-seed#[5F9BD9DF2B6C06E8]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:52175
[junit4] 2> 22227 INFO (IPC Server handler 5 on 33619) [ ] BlockStateChange BLOCK* processReport: from storage DS-074e484e-4a9c-4c5a-a754-a365e030fe9d node DatanodeRegistration(127.0.0.1:60683, datanodeUuid=b84e7b2d-51f3-4470-8267-07ce185b8f8b, infoPort=36793, infoSecurePort=0, ipcPort=60786, storageInfo=lv=-56;cid=testClusterID;nsid=1322244472;c=0), blocks: 0, hasStaleStorage: true, processing time: 7 msecs
[junit4] 2> 22227 INFO (IPC Server handler 5 on 33619) [ ] BlockStateChange BLOCK* processReport: from storage DS-1b09a6e9-7408-4368-a491-d82ccc4da0d3 node DatanodeRegistration(127.0.0.1:60683, datanodeUuid=b84e7b2d-51f3-4470-8267-07ce185b8f8b, infoPort=36793, infoSecurePort=0, ipcPort=60786, storageInfo=lv=-56;cid=testClusterID;nsid=1322244472;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 22248 INFO (IPC Server handler 4 on 33619) [ ] BlockStateChange BLOCK* processReport: from storage DS-3bd95bc6-8696-475c-966e-0727bbf6f2eb node DatanodeRegistration(127.0.0.1:56386, datanodeUuid=af69efc4-09ef-434d-a395-c5ffd09570a0, infoPort=38435, infoSecurePort=0, ipcPort=60230, storageInfo=lv=-56;cid=testClusterID;nsid=1322244472;c=0), blocks: 0, hasStaleStorage: true, processing time: 18 msecs
[junit4] 2> 22263 INFO (IPC Server handler 4 on 33619) [ ] BlockStateChange BLOCK* processReport: from storage DS-a420956c-8fec-46a4-aaff-e572179bbe3c node DatanodeRegistration(127.0.0.1:56386, datanodeUuid=af69efc4-09ef-434d-a395-c5ffd09570a0, infoPort=38435, infoSecurePort=0, ipcPort=60230, storageInfo=lv=-56;cid=testClusterID;nsid=1322244472;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 22654 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 22656 INFO (Thread-115) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 22657 INFO (Thread-115) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 22756 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.c.ZkTestServer start zk server on port:42541
[junit4] 2> 22769 ERROR (Thread-115) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 22893 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15c7dad1c850000, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 22916 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 22932 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 22935 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 22939 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 22942 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 22946 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 22952 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 22958 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 22961 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 22965 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 22972 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 22977 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use TLOG replicas unless explicitly asked otherwise
[junit4] 2> 23542 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_5F9BD9DF2B6C06E8-001/control-001/cores/collection1
[junit4] 2> 23770 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 23835 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@296a3a26{/,null,AVAILABLE}
[junit4] 2> 23846 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@16100155{HTTP/1.1,[http/1.1]}{127.0.0.1:58702}
[junit4] 2> 23846 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.e.j.s.Server Started @26840ms
[junit4] 2> 23846 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:33619/hdfs__localhost_33619__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-master_checkout_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsBasicDistributedZk2Test_5F9BD9DF2B6C06E8-001_tempDir-002_control_data, replicaType=NRT, hostContext=/, hostPort=58702, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_5F9BD9DF2B6C06E8-001/control-001/cores}
[junit4] 2> 23879 ERROR (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 23880 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 23880 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 23880 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 23881 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-06-06T13:51:57.192Z
[junit4] 2> 23883 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
[junit4] 2> 23895 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 23898 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_5F9BD9DF2B6C06E8-001/control-001/solr.xml
[junit4] 2> 23910 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@6dfd12da, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 23934 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42541/solr
[junit4] 2> 24090 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [n:127.0.0.1:58702_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 24092 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [n:127.0.0.1:58702_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:58702_
[junit4] 2> 24096 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [n:127.0.0.1:58702_ ] o.a.s.c.Overseer Overseer (id=98091474356600836-127.0.0.1:58702_-n_0000000000) starting
[junit4] 2> 24172 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [n:127.0.0.1:58702_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58702_
[junit4] 2> 24184 INFO (zkCallback-13-thread-1-processing-n:127.0.0.1:58702_) [n:127.0.0.1:58702_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 24339 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [n:127.0.0.1:58702_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6dfd12da
[junit4] 2> 24340 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [n:127.0.0.1:58702_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6dfd12da
[junit4] 2> 24341 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [n:127.0.0.1:58702_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6dfd12da
[junit4] 2> 24371 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [n:127.0.0.1:58702_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_5F9BD9DF2B6C06E8-001/control-001/cores
[junit4] 2> 24372 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [n:127.0.0.1:58702_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 24383 INFO (coreLoadExecutor-46-thread-1-processing-n:127.0.0.1:58702_) [n:127.0.0.1:58702_ c:control_collection x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 24450 INFO (OverseerStateUpdate-98091474356600836-127.0.0.1:58702_-n_0000000000) [n:127.0.0.1:58702_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 25457 INFO (coreLoadExecutor-46-thread-1-processing-n:127.0.0.1:58702_) [n:127.0.0.1:58702_ c:control_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 25485 INFO (coreLoadExecutor-46-thread-1-processing-n:127.0.0.1:58702_) [n:127.0.0.1:58702_ c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 25623 INFO (coreLoadExecutor-46-thread-1-processing-n:127.0.0.1:58702_) [n:127.0.0.1:58702_ c:control_collection x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 25663 INFO (coreLoadExecutor-46-thread-1-processing-n:127.0.0.1:58702_) [n:127.0.0.1:58702_ c:control_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection, trusted=true
[junit4] 2> 25670 INFO (coreLoadExecutor-46-thread-1-processing-n:127.0.0.1:58702_) [n:127.0.0.1:58702_ c:control_collection x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.core_node1' (registry 'solr.core.control_collection.shard1.core_node1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6dfd12da
[junit4] 2> 25680 INFO (coreLoadExecutor-46-thread-1-processing-n:127.0.0.1:58702_) [n:127.0.0.1:58702_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:33619/solr_hdfs_home
[junit4] 2> 25680 INFO (coreLoadExecutor-46-thread-1-processing-n:127.0.0.1:58702_) [n:127.0.0.1:58702_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 25680 INFO (coreLoadExecutor-46-thread-1-processing-n:127.0.0.1:58702_) [n:127.0.0.1:58702_ c:control_collection x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 25683 INFO (coreLoadExecutor-46-thread-1-processing-n:127.0.0.1:58702_) [n:127.0.0.1:58702_ c:control_collection x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_5F9BD9DF2B6C06E8-001/control-001/cores/collection1], dataDir=[hdfs://localhost:33619/solr_hdfs_home/control_collection/core_node1/data/]
[junit4] 2> 25703 INFO (coreLoadExecutor-46-thread-1-processing-n:127.0.0.1:58702_) [n:127.0.0.1:58702_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:33619/solr_hdfs_home/control_collection/core_node1/data/snapshot_metadata
[junit4] 2> 25781 INFO (coreLoadExecutor-46-thread-1-processing-n:127.0.0.1:58702_) [n:127.0.0.1:58702_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 25782 INFO (coreLoadExecutor-46-thread-1-processing-n:127.0.0.1:58702_) [n:127.0.0.1:58702_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 25782 INFO (coreLoadExecutor-46-thread-1-processing-n:127.0.0.1:58702_) [n:127.0.0.1:58702_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 26711 INFO (coreLoadExecutor-46-thread-1-processing-n:127.0.0.1:58702_) [n:127.0.0.1:58702_ c:control_collection x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 26720 INFO (coreLoadExecutor-46-thread-1-processing-n:127.0.0.1:58702_) [n:127.0.0.1:58702_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:33619/solr_hdfs_home/control_collection/core_node1/data
[junit4] 2> 26767 INFO (coreLoadExecutor-46-thread-1-processing-n:127.0.0.1:58702_) [n:127.0.0.1:58702_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:33619/solr_hdfs_home/control_collection/core_node1/data/index
[junit4] 2> 26777 INFO (coreLoadExecutor-46-thread-1-processing-n:127.0.0.1:58702_) [n:127.0.0.1:58702_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 26777 INFO (coreLoadExecutor-46-thread-1-processing-n:127.0.0.1:58702_) [n:127.0.0.1:58702_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 26777 INFO (coreLoadExecutor-46-thread-1-processing-n:127.0.0.1:58702_) [n:127.0.0.1:58702_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 26790 INFO (coreLoadExecutor-46-thread-1-processing-n:127.0.0.1:58702_) [n:127.0.0.1:58702_ c:control_collection x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 26791 INFO (coreLoadExecutor-46-thread-1-processing-n:127.0.0.1:58702_) [n:127.0.0.1:58702_ c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=13, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8983349364573892]
[junit4] 2> 27095 INFO (IPC Server handler 1 on 33619) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:56386 is added to blk_1073741825_1001{UCState=COMMITTED, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-3bd95bc6-8696-475c-966e-0727bbf6f2eb:NORMAL:127.0.0.1:56386|RBW], ReplicaUC[[DISK]DS-074e484e-4a9c-4c5a-a754-a365e030fe9d:NORMAL:127.0.0.1:60683|RBW]]} size 72
[junit4] 2> 27096 INFO (IPC Server handler 0 on 33619) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:60683 is added to blk_1073741825_1001 size 72
[junit4] 2> 27521 WARN (coreLoadExecutor-46-thread-1-processing-n:127.0.0.1:58702_) [n:127.0.0.1:58702_ c:control_collection x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 27638 INFO (coreLoadExecutor-46-thread-1-processing-n:127.0.0.1:58702_) [n:127.0.0.1:58702_ c:control_collection x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 27638 INFO (coreLoadExecutor-46-thread-1-processing-n:127.0.0.1:58702_) [n:127.0.0.1:58702_ c:control_collection x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 27638 INFO (coreLoadExecutor-46-thread-1-processing-n:127.0.0.1:58702_) [n:127.0.0.1:58702_ c:control_collection x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 27674 INFO (coreLoadExecutor-46-thread-1-processing-n:127.0.0.1:58702_) [n:127.0.0.1:58702_ c:control_collection x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 27675 INFO (coreLoadExecutor-46-thread-1-processing-n:127.0.0.1:58702_) [n:127.0.0.1:58702_ c:control_collection x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 27681 INFO (coreLoadExecutor-46-thread-1-processing-n:127.0.0.1:58702_) [n:127.0.0.1:58702_ c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=41, maxMergeAtOnceExplicit=35, maxMergedSegmentMB=85.1875, floorSegmentMB=1.91796875, forceMergeDeletesPctAllowed=2.62817714184839, segmentsPerTier=24.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 27782 INFO (coreLoadExecutor-46-thread-1-processing-n:127.0.0.1:58702_) [n:127.0.0.1:58702_ c:control_collection x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5c2a156f[collection1] main]
[junit4] 2> 27791 INFO (coreLoadExecutor-46-thread-1-processing-n:127.0.0.1:58702_) [n:127.0.0.1:58702_ c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 27793 INFO (coreLoadExecutor-46-thread-1-processing-n:127.0.0.1:58702_) [n:127.0.0.1:58702_ c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 27798 INFO (coreLoadExecutor-46-thread-1-processing-n:127.0.0.1:58702_) [n:127.0.0.1:58702_ c:control_collection x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 27807 INFO (searcherExecutor-47-thread-1-processing-n:127.0.0.1:58702_ x:collection1 c:control_collection) [n:127.0.0.1:58702_ c:control_collection x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5c2a156f[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 27807 INFO (coreLoadExecutor-46-thread-1-processing-n:127.0.0.1:58702_) [n:127.0.0.1:58702_ c:control_collection x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1569463595033427968
[junit4] 2> 27835 INFO (coreZkRegister-39-thread-1-processing-n:127.0.0.1:58702_ x:collection1 c:control_collection) [n:127.0.0.1:58702_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 27835 INFO (coreZkRegister-39-thread-1-processing-n:127.0.0.1:58702_ x:collection1 c:control_collection) [n:127.0.0.1:58702_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 27838 INFO (coreZkRegister-39-thread-1-processing-n:127.0.0.1:58702_ x:collection1 c:control_collection) [n:127.0.0.1:58702_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:58702/collection1/
[junit4] 2> 27839 INFO (coreZkRegister-39-thread-1-processing-n:127.0.0.1:58702_ x:collection1 c:control_collection) [n:127.0.0.1:58702_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 27840 INFO (coreZkRegister-39-thread-1-processing-n:127.0.0.1:58702_ x:collection1 c:control_collection) [n:127.0.0.1:58702_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:58702/collection1/ has no replicas
[junit4] 2> 27840 INFO (coreZkRegister-39-thread-1-processing-n:127.0.0.1:58702_ x:collection1 c:control_collection) [n:127.0.0.1:58702_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 27869 INFO (coreZkRegister-39-thread-1-processing-n:127.0.0.1:58702_ x:collection1 c:control_collection) [n:127.0.0.1:58702_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:58702/collection1/ shard1
[junit4] 2> 28021 INFO (coreZkRegister-39-thread-1-processing-n:127.0.0.1:58702_ x:collection1 c:control_collection) [n:127.0.0.1:58702_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 28050 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 28057 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42541/solr ready
[junit4] 2> 28059 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 28184 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_5F9BD9DF2B6C06E8-001/shard-1-001/cores/collection1
[junit4] 2> 28185 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_5F9BD9DF2B6C06E8-001/shard-1-001 of type TLOG
[junit4] 2> 28186 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 28188 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@61e4bb69{/,null,AVAILABLE}
[junit4] 2> 28189 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@6ca94693{HTTP/1.1,[http/1.1]}{127.0.0.1:60025}
[junit4] 2> 28190 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.e.j.s.Server Started @31184ms
[junit4] 2> 28190 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:33619/hdfs__localhost_33619__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-master_checkout_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsBasicDistributedZk2Test_5F9BD9DF2B6C06E8-001_tempDir-002_jetty1, replicaType=TLOG, solrconfig=solrconfig.xml, hostContext=/, hostPort=60025, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_5F9BD9DF2B6C06E8-001/shard-1-001/cores}
[junit4] 2> 28190 ERROR (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 28191 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 28191 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 28191 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 28191 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-06-06T13:52:01.502Z
[junit4] 2> 28208 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 28208 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_5F9BD9DF2B6C06E8-001/shard-1-001/solr.xml
[junit4] 2> 28216 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@6dfd12da, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 28220 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42541/solr
[junit4] 2> 28248 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [n:127.0.0.1:60025_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 28252 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [n:127.0.0.1:60025_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 28258 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [n:127.0.0.1:60025_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60025_
[junit4] 2> 28262 INFO (zkCallback-17-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 28263 INFO (zkCallback-13-thread-1-processing-n:127.0.0.1:58702_) [n:127.0.0.1:58702_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 28264 INFO (zkCallback-22-thread-1-processing-n:127.0.0.1:60025_) [n:127.0.0.1:60025_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 28386 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [n:127.0.0.1:60025_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6dfd12da
[junit4] 2> 28399 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [n:127.0.0.1:60025_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6dfd12da
[junit4] 2> 28400 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [n:127.0.0.1:60025_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6dfd12da
[junit4] 2> 28403 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [n:127.0.0.1:60025_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_5F9BD9DF2B6C06E8-001/shard-1-001/cores
[junit4] 2> 28403 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [n:127.0.0.1:60025_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 28406 INFO (coreLoadExecutor-57-thread-1-processing-n:127.0.0.1:60025_) [n:127.0.0.1:60025_ c:collection1 x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 28411 INFO (OverseerStateUpdate-98091474356600836-127.0.0.1:58702_-n_0000000000) [n:127.0.0.1:58702_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 29445 INFO (coreLoadExecutor-57-thread-1-processing-n:127.0.0.1:60025_) [n:127.0.0.1:60025_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 29479 INFO (coreLoadExecutor-57-thread-1-processing-n:127.0.0.1:60025_) [n:127.0.0.1:60025_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 29692 INFO (coreLoadExecutor-57-thread-1-processing-n:127.0.0.1:60025_) [n:127.0.0.1:60025_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 29723 INFO (coreLoadExecutor-57-thread-1-processing-n:127.0.0.1:60025_) [n:127.0.0.1:60025_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1, trusted=true
[junit4] 2> 29724 INFO (coreLoadExecutor-57-thread-1-processing-n:127.0.0.1:60025_) [n:127.0.0.1:60025_ c:collection1 x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.core_node1' (registry 'solr.core.collection1.shard2.core_node1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6dfd12da
[junit4] 2> 29724 INFO (coreLoadExecutor-57-thread-1-processing-n:127.0.0.1:60025_) [n:127.0.0.1:60025_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:33619/solr_hdfs_home
[junit4] 2> 29727 INFO (coreLoadExecutor-57-thread-1-processing-n:127.0.0.1:60025_) [n:127.0.0.1:60025_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 29727 INFO (coreLoadExecutor-57-thread-1-processing-n:127.0.0.1:60025_) [n:127.0.0.1:60025_ c:collection1 x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 29727 INFO (coreLoadExecutor-57-thread-1-processing-n:127.0.0.1:60025_) [n:127.0.0.1:60025_ c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_5F9BD9DF2B6C06E8-001/shard-1-001/cores/collection1], dataDir=[hdfs://localhost:33619/solr_hdfs_home/collection1/core_node1/data/]
[junit4] 2> 29730 INFO (coreLoadExecutor-57-thread-1-processing-n:127.0.0.1:60025_) [n:127.0.0.1:60025_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:33619/solr_hdfs_home/collection1/core_node1/data/snapshot_metadata
[junit4] 2> 29768 INFO (coreLoadExecutor-57-thread-1-processing-n:127.0.0.1:60025_) [n:127.0.0.1:60025_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 29776 INFO (coreLoadExecutor-57-thread-1-processing-n:127.0.0.1:60025_) [n:127.0.0.1:60025_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 29776 INFO (coreLoadExecutor-57-thread-1-processing-n:127.0.0.1:60025_) [n:127.0.0.1:60025_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 29791 INFO (coreLoadExecutor-57-thread-1-processing-n:127.0.0.1:60025_) [n:127.0.0.1:60025_ c:collection1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 29794 INFO (coreLoadExecutor-57-thread-1-processing-n:127.0.0.1:60025_) [n:127.0.0.1:60025_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:33619/solr_hdfs_home/collection1/core_node1/data
[junit4] 2> 29889 INFO (coreLoadExecutor-57-thread-1-processing-n:127.0.0.1:60025_) [n:127.0.0.1:60025_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:33619/solr_hdfs_home/collection1/core_node1/data/index
[junit4] 2> 29902 INFO (coreLoadExecutor-57-thread-1-processing-n:127.0.0.1:60025_) [n:127.0.0.1:60025_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 29902 INFO (coreLoadExecutor-57-thread-1-processing-n:127.0.0.1:60025_) [n:127.0.0.1:60025_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 29902 INFO (coreLoadExecutor-57-thread-1-processing-n:127.0.0.1:60025_) [n:127.0.0.1:60025_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 29948 INFO (coreLoadExecutor-57-thread-1-processing-n:127.0.0.1:60025_) [n:127.0.0.1:60025_ c:collection1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 29949 INFO (coreLoadExecutor-57-thread-1-processing-n:127.0.0.1:60025_) [n:127.0.0.1:60025_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=13, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8983349364573892]
[junit4] 2> 30081 INFO (IPC Server handler 1 on 33619) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:60683 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-1b09a6e9-7408-4368-a491-d82ccc4da0d3:NORMAL:127.0.0.1:60683|RBW], ReplicaUC[[DISK]DS-a420956c-8fec-46a4-aaff-e572179bbe3c:NORMAL:127.0.0.1:56386|RBW]]} size 0
[junit4] 2> 30082 INFO (IPC Server handler 6 on 33619) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:56386 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-1b09a6e9-7408-4368-a491-d82ccc4da0d3:NORMAL:127.0.0.1:60683|RBW], ReplicaUC[[DISK]DS-a420956c-8fec-46a4-aaff-e572179bbe3c:NORMAL:127.0.0.1:56386|RBW]]} size 0
[junit4] 2> 30112 WARN (coreLoadExecutor-57-thread-1-processing-n:127.0.0.1:60025_) [n:127.0.0.1:60025_ c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 30221 INFO (coreLoadExecutor-57-thread-1-processing-n:127.0.0.1:60025_) [n:127.0.0.1:60025_ c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 30221 INFO (coreLoadExecutor-57-thread-1-processing-n:127.0.0.1:60025_) [n:127.0.0.1:60025_ c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 30221 INFO (coreLoadExecutor-57-thread-1-processing-n:127.0.0.1:60025_) [n:127.0.0.1:60025_ c:collection1 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 30238 INFO (coreLoadExecutor-57-thread-1-processing-n:127.0.0.1:60025_) [n:127.0.0.1:60025_ c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 30238 INFO (coreLoadExecutor-57-thread-1-processing-n:127.0.0.1:60025_) [n:127.0.0.1:60025_ c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 30270 INFO (coreLoadExecutor-57-thread-1-processing-n:127.0.0.1:60025_) [n:127.0.0.1:60025_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=41, maxMergeAtOnceExplicit=35, maxMergedSegmentMB=85.1875, floorSegmentMB=1.91796875, forceMergeDeletesPctAllowed=2.62817714184839, segmentsPerTier=24.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 30297 INFO (coreLoadExecutor-57-thread-1-processing-n:127.0.0.1:60025_) [n:127.0.0.1:60025_ c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3f26106b[collection1] main]
[junit4] 2> 30304 INFO (coreLoadExecutor-57-thread-1-processing-n:127.0.0.1:60025_) [n:127.0.0.1:60025_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 30305 INFO (coreLoadExecutor-57-thread-1-processing-n:127.0.0.1:60025_) [n:127.0.0.1:60025_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 30306 INFO (coreLoadExecutor-57-thread-1-processing-n:127.0.0.1:60025_) [n:127.0.0.1:60025_ c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 30310 INFO (searcherExecutor-58-thread-1-processing-n:127.0.0.1:60025_ x:collection1 c:collection1) [n:127.0.0.1:60025_ c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@3f26106b[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 30311 INFO (coreLoadExecutor-57-thread-1-processing-n:127.0.0.1:60025_) [n:127.0.0.1:60025_ c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1569463597659062272
[junit4] 2> 30345 INFO (coreZkRegister-52-thread-1-processing-n:127.0.0.1:60025_ x:collection1 c:collection1) [n:127.0.0.1:60025_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 30345 INFO (coreZkRegister-52-thread-1-processing-n:127.0.0.1:60025_ x:collection1 c:collection1) [n:127.0.0.1:60025_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 30345 INFO (coreZkRegister-52-thread-1-processing-n:127.0.0.1:60025_ x:collection1 c:collection1) [n:127.0.0.1:60025_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:60025/collection1/
[junit4] 2> 30345 INFO (coreZkRegister-52-thread-1-processing-n:127.0.0.1:60025_ x:collection1 c:collection1) [n:127.0.0.1:60025_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 30345 INFO (coreZkRegister-52-thread-1-processing-n:127.0.0.1:60025_ x:collection1 c:collection1) [n:127.0.0.1:60025_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:60025/collection1/ has no replicas
[junit4] 2> 30346 INFO (coreZkRegister-52-thread-1-processing-n:127.0.0.1:60025_ x:collection1 c:collection1) [n:127.0.0.1:60025_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 30346 INFO (coreZkRegister-52-thread-1-processing-n:127.0.0.1:60025_ x:collection1 c:collection1) [n:127.0.0.1:60025_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController collection1 stopping background replication from leader
[junit4] 2> 30354 INFO (coreZkRegister-52-thread-1-processing-n:127.0.0.1:60025_ x:collection1 c:collection1) [n:127.0.0.1:60025_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:60025/collection1/ shard2
[junit4] 2> 30507 INFO (coreZkRegister-52-thread-1-processing-n:127.0.0.1:60025_ x:collection1 c:collection1) [n:127.0.0.1:60025_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 30615 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_5F9BD9DF2B6C06E8-001/shard-2-001/cores/collection1
[junit4] 2> 30616 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_5F9BD9DF2B6C06E8-001/shard-2-001 of type TLOG
[junit4] 2> 30617 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 30619 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@40bec135{/,null,AVAILABLE}
[junit4] 2> 30620 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@53e2b60b{HTTP/1.1,[http/1.1]}{127.0.0.1:59692}
[junit4] 2> 30620 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.e.j.s.Server Started @33614ms
[junit4] 2> 30620 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:33619/hdfs__localhost_33619__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-master_checkout_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsBasicDistributedZk2Test_5F9BD9DF2B6C06E8-001_tempDir-002_jetty2, replicaType=TLOG, solrconfig=solrconfig.xml, hostContext=/, hostPort=59692, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_5F9BD9DF2B6C06E8-001/shard-2-001/cores}
[junit4] 2> 30621 ERROR (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 30624 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 30624 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 30624 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 30624 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-06-06T13:52:03.935Z
[junit4] 2> 30639 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 30639 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_5F9BD9DF2B6C06E8-001/shard-2-001/solr.xml
[junit4] 2> 30656 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@6dfd12da, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 30660 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42541/solr
[junit4] 2> 30730 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [n:127.0.0.1:59692_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 30736 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [n:127.0.0.1:59692_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 30740 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [n:127.0.0.1:59692_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59692_
[junit4] 2> 30743 INFO (zkCallback-17-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 30743 INFO (zkCallback-22-thread-1-processing-n:127.0.0.1:60025_) [n:127.0.0.1:60025_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 30744 INFO (zkCallback-13-thread-2-processing-n:127.0.0.1:58702_) [n:127.0.0.1:58702_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 30746 INFO (zkCallback-28-thread-1-processing-n:127.0.0.1:59692_) [n:127.0.0.1:59692_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 30944 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [n:127.0.0.1:59692_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6dfd12da
[junit4] 2> 30953 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [n:127.0.0.1:59692_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6dfd12da
[junit4] 2> 30953 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [n:127.0.0.1:59692_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6dfd12da
[junit4] 2> 30956 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [n:127.0.0.1:59692_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_5F9BD9DF2B6C06E8-001/shard-2-001/cores
[junit4] 2> 30957 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [n:127.0.0.1:59692_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 30970 INFO (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:59692_) [n:127.0.0.1:59692_ c:collection1 x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 30975 INFO (OverseerStateUpdate-98091474356600836-127.0.0.1:58702_-n_0000000000) [n:127.0.0.1:58702_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 32013 INFO (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:59692_) [n:127.0.0.1:59692_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 32039 INFO (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:59692_) [n:127.0.0.1:59692_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 32206 INFO (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:59692_) [n:127.0.0.1:59692_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 32240 INFO (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:59692_) [n:127.0.0.1:59692_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1, trusted=true
[junit4] 2> 32241 INFO (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:59692_) [n:127.0.0.1:59692_ c:collection1 x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.core_node2' (registry 'solr.core.collection1.shard1.core_node2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6dfd12da
[junit4] 2> 32241 INFO (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:59692_) [n:127.0.0.1:59692_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:33619/solr_hdfs_home
[junit4] 2> 32241 INFO (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:59692_) [n:127.0.0.1:59692_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 32241 INFO (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:59692_) [n:127.0.0.1:59692_ c:collection1 x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 32242 INFO (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:59692_) [n:127.0.0.1:59692_ c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_5F9BD9DF2B6C06E8-001/shard-2-001/cores/collection1], dataDir=[hdfs://localhost:33619/solr_hdfs_home/collection1/core_node2/data/]
[junit4] 2> 32244 INFO (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:59692_) [n:127.0.0.1:59692_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:33619/solr_hdfs_home/collection1/core_node2/data/snapshot_metadata
[junit4] 2> 32265 INFO (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:59692_) [n:127.0.0.1:59692_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 32265 INFO (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:59692_) [n:127.0.0.1:59692_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 32265 INFO (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:59692_) [n:127.0.0.1:59692_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 32282 INFO (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:59692_) [n:127.0.0.1:59692_ c:collection1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 32284 INFO (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:59692_) [n:127.0.0.1:59692_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:33619/solr_hdfs_home/collection1/core_node2/data
[junit4] 2> 32328 INFO (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:59692_) [n:127.0.0.1:59692_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:33619/solr_hdfs_home/collection1/core_node2/data/index
[junit4] 2> 32338 INFO (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:59692_) [n:127.0.0.1:59692_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 32339 INFO (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:59692_) [n:127.0.0.1:59692_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 32339 INFO (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:59692_) [n:127.0.0.1:59692_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 32350 INFO (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:59692_) [n:127.0.0.1:59692_ c:collection1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 32352 INFO (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:59692_) [n:127.0.0.1:59692_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=13, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8983349364573892]
[junit4] 2> 32397 INFO (IPC Server handler 5 on 33619) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:60683 is added to blk_1073741827_1003{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-a420956c-8fec-46a4-aaff-e572179bbe3c:NORMAL:127.0.0.1:56386|RBW], ReplicaUC[[DISK]DS-074e484e-4a9c-4c5a-a754-a365e030fe9d:NORMAL:127.0.0.1:60683|RBW]]} size 0
[junit4] 2> 32400 INFO (IPC Server handler 7 on 33619) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:56386 is added to blk_1073741827_1003{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-074e484e-4a9c-4c5a-a754-a365e030fe9d:NORMAL:127.0.0.1:60683|RBW], ReplicaUC[[DISK]DS-3bd95bc6-8696-475c-966e-0727bbf6f2eb:NORMAL:127.0.0.1:56386|FINALIZED]]} size 0
[junit4] 2> 32414 WARN (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:59692_) [n:127.0.0.1:59692_ c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 32500 INFO (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:59692_) [n:127.0.0.1:59692_ c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 32500 INFO (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:59692_) [n:127.0.0.1:59692_ c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 32500 INFO (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:59692_) [n:127.0.0.1:59692_ c:collection1 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 32530 INFO (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:59692_) [n:127.0.0.1:59692_ c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 32530 INFO (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:59692_) [n:127.0.0.1:59692_ c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 32547 INFO (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:59692_) [n:127.0.0.1:59692_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=41, maxMergeAtOnceExplicit=35, maxMergedSegmentMB=85.1875, floorSegmentMB=1.91796875, forceMergeDeletesPctAllowed=2.62817714184839, segmentsPerTier=24.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 32576 INFO (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:59692_) [n:127.0.0.1:59692_ c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@338e8ed6[collection1] main]
[junit4] 2> 32578 INFO (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:59692_) [n:127.0.0.1:59692_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 32579 INFO (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:59692_) [n:127.0.0.1:59692_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 32580 INFO (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:59692_) [n:127.0.0.1:59692_ c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 32584 INFO (searcherExecutor-69-thread-1-processing-n:127.0.0.1:59692_ x:collection1 c:collection1) [n:127.0.0.1:59692_ c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@338e8ed6[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 32584 INFO (coreLoadExecutor-68-thread-1-processing-n:127.0.0.1:59692_) [n:127.0.0.1:59692_ c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1569463600042475520
[junit4] 2> 32605 INFO (coreZkRegister-63-thread-1-processing-n:127.0.0.1:59692_ x:collection1 c:collection1) [n:127.0.0.1:59692_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 32608 INFO (coreZkRegister-63-thread-1-processing-n:127.0.0.1:59692_ x:collection1 c:collection1) [n:127.0.0.1:59692_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 32608 INFO (coreZkRegister-63-thread-1-processing-n:127.0.0.1:59692_ x:collection1 c:collection1) [n:127.0.0.1:59692_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:59692/collection1/
[junit4] 2> 32609 INFO (coreZkRegister-63-thread-1-processing-n:127.0.0.1:59692_ x:collection1 c:collection1) [n:127.0.0.1:59692_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 32609 INFO (coreZkRegister-63-thread-1-processing-n:127.0.0.1:59692_ x:collection1 c:collection1) [n:127.0.0.1:59692_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:59692/collection1/ has no replicas
[junit4] 2> 32609 INFO (coreZkRegister-63-thread-1-processing-n:127.0.0.1:59692_ x:collection1 c:collection1) [n:127.0.0.1:59692_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 32610 INFO (coreZkRegister-63-thread-1-processing-n:127.0.0.1:59692_ x:collection1 c:collection1) [n:127.0.0.1:59692_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ZkController collection1 stopping background replication from leader
[junit4] 2> 32615 INFO (coreZkRegister-63-thread-1-processing-n:127.0.0.1:59692_ x:collection1 c:collection1) [n:127.0.0.1:59692_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:59692/collection1/ shard1
[junit4] 2> 32766 INFO (coreZkRegister-63-thread-1-processing-n:127.0.0.1:59692_ x:collection1 c:collection1) [n:127.0.0.1:59692_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 33218 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_5F9BD9DF2B6C06E8-001/shard-3-001/cores/collection1
[junit4] 2> 33230 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_5F9BD9DF2B6C06E8-001/shard-3-001 of type TLOG
[junit4] 2> 33234 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 33235 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4b5335ad{/,null,AVAILABLE}
[junit4] 2> 33236 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7fa18d27{HTTP/1.1,[http/1.1]}{127.0.0.1:52773}
[junit4] 2> 33236 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.e.j.s.Server Started @36230ms
[junit4] 2> 33236 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:33619/hdfs__localhost_33619__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-master_checkout_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsBasicDistributedZk2Test_5F9BD9DF2B6C06E8-001_tempDir-002_jetty3, replicaType=TLOG, solrconfig=solrconfig.xml, hostContext=/, hostPort=52773, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_5F9BD9DF2B6C06E8-001/shard-3-001/cores}
[junit4] 2> 33237 ERROR (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 33237 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 33237 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 33238 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 33238 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-06-06T13:52:06.549Z
[junit4] 2> 33262 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 33262 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_5F9BD9DF2B6C06E8-001/shard-3-001/solr.xml
[junit4] 2> 33293 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@6dfd12da, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 33301 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42541/solr
[junit4] 2> 33333 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [n:127.0.0.1:52773_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 33337 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [n:127.0.0.1:52773_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 33340 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [n:127.0.0.1:52773_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52773_
[junit4] 2> 33342 INFO (zkCallback-13-thread-2-processing-n:127.0.0.1:58702_) [n:127.0.0.1:58702_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 33348 INFO (zkCallback-28-thread-1-processing-n:127.0.0.1:59692_) [n:127.0.0.1:59692_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 33348 INFO (zkCallback-17-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 33349 INFO (zkCallback-22-thread-1-processing-n:127.0.0.1:60025_) [n:127.0.0.1:60025_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 33352 INFO (zkCallback-34-thread-1-processing-n:127.0.0.1:52773_) [n:127.0.0.1:52773_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 33452 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [n:127.0.0.1:52773_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6dfd12da
[junit4] 2> 33463 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [n:127.0.0.1:52773_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6dfd12da
[junit4] 2> 33463 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [n:127.0.0.1:52773_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6dfd12da
[junit4] 2> 33466 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [n:127.0.0.1:52773_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_5F9BD9DF2B6C06E8-001/shard-3-001/cores
[junit4] 2> 33467 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[5F9BD9DF2B6C06E8]) [n:127.0.0.1:52773_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 33470 INFO (coreLoadExecutor-79-thread-1-processing-n:127.0.0.1:52773_) [n:127.0.0.1:52773_ c:collection1 x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 33476 INFO (OverseerStateUpdate-98091474356600836-127.0.0.1:58702_-n_0000000000) [n:127.0.0.1:58702_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 34515 INFO (coreLoadExecutor-79-thread-1-processing-n:127.0.0.1:52773_) [n:127.0.0.1:52773_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 34570 INFO (coreLoadExecutor-79-thread-1-processing-n:127.0.0.1:52773_) [n:127.0.0.1:52773_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 34764 INFO (coreLoadExecutor-79-thread-1-processing-n:127.0.0.1:52773_) [n:127.0.0.1:52773_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 34817 INFO (coreLoadExecutor-79-thread-1-processing-n:127.0.0.1:52773_) [n:127.0.0.1:52773_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1, trusted=true
[junit4] 2> 34817 INFO (coreLoadExecutor-79-thread-1-processing-n:127.0.0.1:52773_) [n:127.0.0.1:52773_ c:collection1 x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.core_node3' (registry 'solr.core.collection1.shard2.core_node3') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6dfd12da
[junit4] 2> 34818 INFO (coreLoadExecutor-79-thread-1-processing-n:127.0.0.1:52773_) [n:127.0.0.1:52773_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:33619/solr_hdfs_home
[junit4] 2> 34818 INFO (coreLoadExecutor-79-thread-1-processing-n:127.0.0.1:52773_) [n:127.0.0.1:52773_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 34818 INFO (coreLoadExecutor-79-thread-1-processing-n:127.0.0.1:52773_) [n:127.0.0.1:52773_ c:collection1 x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 34818 INFO (coreLoadExecutor-79-thread-1-processing-n:127.0.0.1:52773_) [n:127.0.0.1:52773_ c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_5F9BD9DF2B6C06E8-001/shard-3-001/cores/collection1], dataDir=[hdfs://localhost:33619/solr_hdfs_home/collection1/core_node3/data/]
[junit4] 2> 34820 INFO (coreLoadExecutor-79-thread-1-processing-n:127.0.0.1:52773_) [n:127.0.0.1:52773_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:33619/solr_hdfs_home/collection1/core_node3/data/snapshot_metadata
[junit4] 2> 34844 INFO (coreLoadExecutor-79-thread-1-processing-n:127.0.0.1:52773_) [n:127.0.0.1:52773_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 34846 INFO (coreLoadExecutor-79-thread-1-processing-n:127.0.0.1:52773_) [n:127.0.0.1:52773_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 34847 INFO (coreLoadExecutor-79-thread-1-processing-n:127.0.0.1:52773_) [n:127.0.0.1:52773_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 34877 INFO (coreLoadExecutor-79-thread-1-processing-n:127.0.0.1:52773_) [n:127.0.0.1:52773_ c:collection1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 34878 INFO (coreLoadExecutor-79-thread-1-processing-n:127.0.0.1:52773_) [n:127.0.0.1:52773_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:33619/solr_hdfs_home/collection1/core_node3/data
[junit4] 2> 34918 INFO (coreLoadExecutor-79-thread-1-processing-n:127.0.0.1:52773_) [n:127.0.0.1:52773_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:33619/solr_hdfs_home/collection1/core_node3/data/index
[junit4] 2> 34927 INFO (coreLoadExecutor-79-thread-1-processing-n:127.0.0.1:52773_) [n:127.0.0.1:52773_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 34927 INFO (coreLoadExecutor-79-thread-1-processing-n:127.0.0.1:52773_) [n:127.0.0.1:52773_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 34927 INFO (coreLoadExecutor-79-thread-1-processing-n:127.0.0.1:52773_) [n:127.0.0.1:52773_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 34970 INFO (coreLoadExecutor-79-thread-1-processing-n:127.0.0.1:52773_) [n:127.0.0.1:52773_ c:collection1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 34971 INFO (coreLoadExecutor-79-thread-1-processing-n:127.0.0.1:52773_) [n:127.0.0.1:52773_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=13, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8983349364573892]
[junit4] 2> 35086 INFO (IPC Server handler 7 on 33619) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:60683 is added to blk_1073741828_1004{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-3bd95bc6-8696-475c-966e-0727bbf6f2eb:NORMAL:127.0.0.1:56386|RBW], ReplicaUC[[DISK]DS-1b09a6e9-7408-4368-a491-d82ccc4da0d3:NORMAL:127.0.0.1:60683|RBW]]} size 0
[junit4] 2> 35090 INFO (IPC Server handler 0 on 33619) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:56386 is added to blk_1073741828_1004{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-1b09a6e9-7408-4368-a491-d82ccc4da0d3:NORMAL:127.0.0.1:60683|RBW], ReplicaUC[[DISK]DS-a420956c-8fec-46a4-aaff-e572179bbe3c:NORMAL:127.0.0.1:56386|FINALIZED]]} size 0
[junit4] 2> 35124 WARN (coreLoadExecutor-79-thread-1-processing-n:127.0.0.1:52773_) [n:127.0.0.1:52773_ c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 35436 INFO (coreLoadExecutor-79-thread-1-processing-n:127.0.0.1:52773_) [n:127.0.0.1:52773_ c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 35436 INFO (coreLoadExecutor-79-thread-1-processing-n:127.0.0.1:52773_) [n:127.0.0.1:52773_ c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 35436 INFO (coreLoadExecutor-79-thread-1-processing-n:127.0.0.1:52773_) [n:127.0.0.1:52773_ c:collection1 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 35484 INFO (coreLoadExecutor-79-thread-1-processing-n:127.0.0.1:52773_) [n:127.0.0.1:52773_ c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 35485 INFO (coreLoadExecutor-79-thread-1-processing-n:127.0.0.1:52773_) [n:127.0.0.1:52773_ c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 35496 INFO (coreLoadExecutor-79-thread-1-processing-n:127.0.0.1:52773_) [n:127.0.0.1:52773_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=41, maxMergeAtOnceExplicit=35, maxMergedSegmentMB=85.1875, floorSegmentMB=1.91796875, forceMergeDeletesPctAllowed=2.62817714184839, segmentsPerTier=24.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 35560 INFO (coreLoadExecutor-79-thread-1-processing-n:127.0.0.1:52773_) [n:127.0.0.1:52773_ c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1730de8f[collection1] main]
[junit4] 2> 35562 INFO (coreLoadExecutor-79-thread-1-processing-n:127.0.0.1:52773_) [n:127.0.0.1:52773_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 35564 INFO (coreLoadExecutor-79-thread-1-processing-n:127.0.0.1:52773_) [n:127.0.0.1:52773_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 35565 INFO (coreLoadExecutor-79-thread-1-processing-n:127.0.0.1:52773_) [n:127.0.0.1:52773_ c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 35568 INFO (searcherExecutor-80-thread-1-processing-n:127.0.0.1:52773_ x:collection1 c:collection1) [n:127.0.0.1:52773_ c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1730de8f[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 35568 INFO (coreLoadExecutor-79-thread-1-processing-n:127.0.0.1:52773_) [n:127.0.0.1:52773_ c:collection1 x:collection1] o.a
[...truncated too long message...]
6-10.41.0.5-1496757104799 (Datanode Uuid b84e7b2d-51f3-4470-8267-07ce185b8f8b) service to localhost/127.0.0.1:33619 interrupted
[junit4] 2> 96158 WARN (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_5F9BD9DF2B6C06E8-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_5F9BD9DF2B6C06E8-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to localhost/127.0.0.1:33619) [ ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-1634483376-10.41.0.5-1496757104799 (Datanode Uuid b84e7b2d-51f3-4470-8267-07ce185b8f8b) service to localhost/127.0.0.1:33619
[junit4] 2> 96183 INFO (SUITE-HdfsBasicDistributedZk2Test-seed#[5F9BD9DF2B6C06E8]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
[junit4] 2> 96287 ERROR (SUITE-HdfsBasicDistributedZk2Test-seed#[5F9BD9DF2B6C06E8]-worker) [ ] o.a.h.m.l.MethodMetric Error invoking method getBlocksTotal
[junit4] 2> java.lang.reflect.InvocationTargetException
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at org.apache.hadoop.metrics2.lib.MethodMetric$2.snapshot(MethodMetric.java:111)
[junit4] 2> at org.apache.hadoop.metrics2.lib.MethodMetric.snapshot(MethodMetric.java:144)
[junit4] 2> at org.apache.hadoop.metrics2.lib.MetricsRegistry.snapshot(MetricsRegistry.java:401)
[junit4] 2> at org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1.getMetrics(MetricsSourceBuilder.java:79)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMetrics(MetricsSourceAdapter.java:194)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache(MetricsSourceAdapter.java:172)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMBeanInfo(MetricsSourceAdapter.java:151)
[junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getClassName(DefaultMBeanServerInterceptor.java:1804)
[junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.safeGetClassName(DefaultMBeanServerInterceptor.java:1595)
[junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanPermission(DefaultMBeanServerInterceptor.java:1813)
[junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:430)
[junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)
[junit4] 2> at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)
[junit4] 2> at org.apache.hadoop.metrics2.util.MBeans.unregister(MBeans.java:81)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(MetricsSourceAdapter.java:226)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:211)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:463)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:213)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:594)
[junit4] 2> at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
[junit4] 2> at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
[junit4] 2> at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:171)
[junit4] 2> at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:872)
[junit4] 2> at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1726)
[junit4] 2> at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1705)
[junit4] 2> at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:204)
[junit4] 2> at org.apache.solr.cloud.hdfs.HdfsBasicDistributedZk2Test.teardownClass(HdfsBasicDistributedZk2Test.java:46)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> Caused by: java.lang.NullPointerException
[junit4] 2> at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:203)
[junit4] 2> at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3370)
[junit4] 2> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:5729)
[junit4] 2> ... 54 more
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_5F9BD9DF2B6C06E8-001
[junit4] 2> Jun 06, 2017 1:53:09 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 35 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {other_tl1=PostingsFormat(name=LuceneVarGapFixedInterval), SubjectTerms_mfacet=PostingsFormat(name=LuceneVarGapFixedInterval), multiDefault=PostingsFormat(name=Asserting), a_i1=PostingsFormat(name=LuceneVarGapFixedInterval), n_td1=PostingsFormat(name=Asserting), n_ti1=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), rnd_b=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), n_tl1=PostingsFormat(name=Asserting), oddField_s=FST50, n_tf1=PostingsFormat(name=LuceneVarGapFixedInterval), a_t=PostingsFormat(name=LuceneVarGapFixedInterval), foo_b=FST50, n_tdt1=PostingsFormat(name=LuceneVarGapFixedInterval), id=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), text=FST50}, docValues:{range_facet_l_dv=DocValuesFormat(name=Lucene70), n_l1=DocValuesFormat(name=Direct), intDefault=DocValuesFormat(name=Direct), n_dt1=DocValuesFormat(name=Lucene70), n_d1=DocValuesFormat(name=Direct), range_facet_l=DocValuesFormat(name=Direct), n_f1=DocValuesFormat(name=Lucene70), _version_=DocValuesFormat(name=Direct), foo_d=DocValuesFormat(name=Lucene70), range_facet_i_dv=DocValuesFormat(name=Direct), intDvoDefault=DocValuesFormat(name=Memory), foo_f=DocValuesFormat(name=Memory), timestamp=DocValuesFormat(name=Direct)}, maxPointsInLeafNode=1120, maxMBSortInHeap=6.5334892881027375, sim=RandomSimilarity(queryNorm=false): {}, locale=ja, timezone=Pacific/Chuuk
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_121 (64-bit)/cpus=4,threads=5,free=277540968,total=382730240
[junit4] 2> NOTE: All tests run in this JVM: [IndexSchemaTest, TestPartialUpdateDeduplication, SampleTest, HdfsBasicDistributedZk2Test]
[junit4] Completed [24/726 (1!)] on J1 in 96.20s, 1 test, 1 failure <<< FAILURES!
[...truncated 45630 lines...]