You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2017/08/20 10:41:56 UTC
[JENKINS] Lucene-Solr-NightlyTests-6.6 - Build # 24 - Unstable
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.6/24/
10 tests failed.
FAILED: org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomMedium
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([1B7B70F64CC3238E]:0)
FAILED: junit.framework.TestSuite.org.apache.lucene.spatial3d.TestGeo3DPoint
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([1B7B70F64CC3238E]:0)
FAILED: org.apache.solr.cloud.CdcrReplicationHandlerTest.testReplicationWithBufferedUpdates
Error Message:
Timeout while trying to assert number of documents @ source_collection
Stack Trace:
java.lang.AssertionError: Timeout while trying to assert number of documents @ source_collection
at __randomizedtesting.SeedInfo.seed([7C9675E2CB1A59B5:AF9F25FC8E89C522]:0)
at org.apache.solr.cloud.BaseCdcrDistributedZkTest.assertNumDocs(BaseCdcrDistributedZkTest.java:271)
at org.apache.solr.cloud.CdcrReplicationHandlerTest.testReplicationWithBufferedUpdates(CdcrReplicationHandlerTest.java:233)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.AssertionError: expected:<230> but was:<240>
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.BaseCdcrDistributedZkTest.assertNumDocs(BaseCdcrDistributedZkTest.java:262)
... 42 more
FAILED: org.apache.solr.cloud.MissingSegmentRecoveryTest.testLeaderRecovery
Error Message:
Expected a collection with one shard and two replicas null Last available state: DocCollection(MissingSegmentRecoveryTest//collections/MissingSegmentRecoveryTest/state.json/8)={ "replicationFactor":"2", "shards":{"shard1":{ "range":"80000000-7fffffff", "state":"active", "replicas":{ "core_node1":{ "core":"MissingSegmentRecoveryTest_shard1_replica1", "base_url":"http://127.0.0.1:40105/solr", "node_name":"127.0.0.1:40105_solr", "state":"down"}, "core_node2":{ "core":"MissingSegmentRecoveryTest_shard1_replica2", "base_url":"http://127.0.0.1:36729/solr", "node_name":"127.0.0.1:36729_solr", "state":"active", "leader":"true"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"false"}
Stack Trace:
java.lang.AssertionError: Expected a collection with one shard and two replicas
null
Last available state: DocCollection(MissingSegmentRecoveryTest//collections/MissingSegmentRecoveryTest/state.json/8)={
"replicationFactor":"2",
"shards":{"shard1":{
"range":"80000000-7fffffff",
"state":"active",
"replicas":{
"core_node1":{
"core":"MissingSegmentRecoveryTest_shard1_replica1",
"base_url":"http://127.0.0.1:40105/solr",
"node_name":"127.0.0.1:40105_solr",
"state":"down"},
"core_node2":{
"core":"MissingSegmentRecoveryTest_shard1_replica2",
"base_url":"http://127.0.0.1:36729/solr",
"node_name":"127.0.0.1:36729_solr",
"state":"active",
"leader":"true"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false"}
at __randomizedtesting.SeedInfo.seed([7C9675E2CB1A59B5:2CC3EDE1923BEFA8]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:265)
at org.apache.solr.cloud.MissingSegmentRecoveryTest.testLeaderRecovery(MissingSegmentRecoveryTest.java:105)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.SyncSliceTest.test
Error Message:
timeout waiting to see all nodes active
Stack Trace:
java.lang.AssertionError: timeout waiting to see all nodes active
at __randomizedtesting.SeedInfo.seed([7C9675E2CB1A59B5:F4C24A3865E6344D]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.SyncSliceTest.waitTillAllNodesActive(SyncSliceTest.java:235)
at org.apache.solr.cloud.SyncSliceTest.test(SyncSliceTest.java:164)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.UnloadDistributedZkTest.test
Error Message:
KeeperErrorCode = Session expired for /clusterstate.json
Stack Trace:
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /clusterstate.json
at __randomizedtesting.SeedInfo.seed([7C9675E2CB1A59B5:F4C24A3865E6344D]:0)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1212)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:357)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:354)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:354)
at org.apache.solr.common.cloud.ZkStateReader.refreshLegacyClusterState(ZkStateReader.java:532)
at org.apache.solr.common.cloud.ZkStateReader.forceUpdateCollection(ZkStateReader.java:300)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.updateMappingsFromZk(AbstractFullDistribZkTestBase.java:622)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.updateMappingsFromZk(AbstractFullDistribZkTestBase.java:617)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createJettys(AbstractFullDistribZkTestBase.java:430)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:331)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:990)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testCollectionsAPI
Error Message:
Something is broken in the assert for no shards using the same indexDir - probably something was changed in the attributes published in the MBean of SolrCore : {}
Stack Trace:
java.lang.AssertionError: Something is broken in the assert for no shards using the same indexDir - probably something was changed in the attributes published in the MBean of SolrCore : {}
at __randomizedtesting.SeedInfo.seed([7C9675E2CB1A59B5:34E30156CD297620]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.checkNoTwoShardsUseTheSameIndexDir(CollectionsAPIDistributedZkTest.java:646)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:524)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.hdfs.HdfsSyncSliceTest.test
Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:46089/collection1
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:46089/collection1
at __randomizedtesting.SeedInfo.seed([7C9675E2CB1A59B5:F4C24A3865E6344D]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:621)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:447)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:388)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1383)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1134)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1073)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:484)
at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:463)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.commit(AbstractFullDistribZkTestBase.java:1530)
at org.apache.solr.cloud.SyncSliceTest.test(SyncSliceTest.java:183)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:515)
... 53 more
FAILED: org.apache.solr.cloud.hdfs.HdfsUnloadDistributedZkTest.test
Error Message:
Captured an uncaught exception in thread: Thread[id=22826, name=testExecutor-1397-thread-4, state=RUNNABLE, group=TGRP-HdfsUnloadDistributedZkTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=22826, name=testExecutor-1397-thread-4, state=RUNNABLE, group=TGRP-HdfsUnloadDistributedZkTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:37190
at __randomizedtesting.SeedInfo.seed([7C9675E2CB1A59B5]:0)
at org.apache.solr.cloud.UnloadDistributedZkTest.lambda$testUnloadLotsOfCores$0(UnloadDistributedZkTest.java:412)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:37190
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:621)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at org.apache.solr.cloud.UnloadDistributedZkTest.lambda$testUnloadLotsOfCores$0(UnloadDistributedZkTest.java:410)
... 4 more
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:515)
... 8 more
FAILED: org.apache.solr.common.cloud.TestCollectionStateWatchers.testWaitForStateWatcherIsRetainedOnPredicateFailure
Error Message:
Did not see a fully active cluster after 30 seconds
Stack Trace:
java.lang.AssertionError: Did not see a fully active cluster after 30 seconds
at __randomizedtesting.SeedInfo.seed([EAD0DD62EFD59022:62E67F31377A7830]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.common.cloud.TestCollectionStateWatchers.testWaitForStateWatcherIsRetainedOnPredicateFailure(TestCollectionStateWatchers.java:250)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 10414 lines...]
[junit4] Suite: org.apache.lucene.spatial3d.TestGeo3DPoint
[junit4] 2> kol 20, 2017 6:53:54 AM com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
[junit4] 2> WARNING: Suite execution timed out: org.apache.lucene.spatial3d.TestGeo3DPoint
[junit4] 2> 1) Thread[id=1, name=main, state=WAITING, group=main]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Thread.join(Thread.java:1252)
[junit4] 2> at java.lang.Thread.join(Thread.java:1326)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:608)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:457)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:244)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:355)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:13)
[junit4] 2> 2) Thread[id=11, name=SUITE-TestGeo3DPoint-seed#[1B7B70F64CC3238E], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at java.lang.Thread.getStackTrace(Thread.java:1559)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:696)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:693)
[junit4] 2> at java.security.AccessController.doPrivileged(Native Method)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getStackTrace(ThreadLeakControl.java:693)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getThreadsWithTraces(ThreadLeakControl.java:709)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:689)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:65)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:415)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:678)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:140)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:598)
[junit4] 2> 3) Thread[id=12, name=TEST-TestGeo3DPoint.testRandomMedium-seed#[1B7B70F64CC3238E], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at org.apache.lucene.spatial3d.geom.StandardXYZSolid.<init>(StandardXYZSolid.java:145)
[junit4] 2> at org.apache.lucene.spatial3d.geom.XYZSolidFactory.makeXYZSolid(XYZSolidFactory.java:64)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.verify(TestGeo3DPoint.java:846)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.doTestRandom(TestGeo3DPoint.java:548)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomMedium(TestGeo3DPoint.java:475)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 4) Thread[id=9, name=JUnit4-serializer-daemon, state=TIMED_WAITING, group=main]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:50)
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestGeo3DPoint -Dtests.method=testRandomMedium -Dtests.seed=1B7B70F64CC3238E -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/test-data/enwiki.random.lines.txt -Dtests.locale=hr-HR -Dtests.timezone=UCT -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 24.4s J1 | TestGeo3DPoint.testRandomMedium <<<
[junit4] > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
[junit4] > at __randomizedtesting.SeedInfo.seed([1B7B70F64CC3238E]:0)
[junit4] 2> kol 20, 2017 6:53:54 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> kol 20, 2017 6:54:15 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.lucene.spatial3d.TestGeo3DPoint:
[junit4] 2> 1) Thread[id=12, name=TEST-TestGeo3DPoint.testRandomMedium-seed#[1B7B70F64CC3238E], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at org.apache.lucene.spatial3d.geom.Plane.findIntersections(Plane.java:650)
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoComplexPolygon.isWithin(GeoComplexPolygon.java:164)
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoBaseMembershipShape.isWithin(GeoBaseMembershipShape.java:36)
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoBaseShape.getBounds(GeoBaseShape.java:35)
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoComplexPolygon.getBounds(GeoComplexPolygon.java:375)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.verify(TestGeo3DPoint.java:845)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.doTestRandom(TestGeo3DPoint.java:548)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomMedium(TestGeo3DPoint.java:475)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> kol 20, 2017 6:54:15 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=12, name=TEST-TestGeo3DPoint.testRandomMedium-seed#[1B7B70F64CC3238E], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> kol 20, 2017 6:54:18 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> SEVERE: There are still zombie threads that couldn't be terminated:
[junit4] 2> 1) Thread[id=12, name=TEST-TestGeo3DPoint.testRandomMedium-seed#[1B7B70F64CC3238E], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at org.apache.lucene.spatial3d.geom.Plane.findIntersections(Plane.java:650)
[junit4] 2> at org.apache.lucene.spatial3d.geom.StandardXYZSolid.<init>(StandardXYZSolid.java:164)
[junit4] 2> at org.apache.lucene.spatial3d.geom.XYZSolidFactory.makeXYZSolid(XYZSolidFactory.java:64)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.verify(TestGeo3DPoint.java:846)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.doTestRandom(TestGeo3DPoint.java:548)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomMedium(TestGeo3DPoint.java:475)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): {id=Lucene50(blocksize=128)}, docValues:{id=DocValuesFormat(name=Direct), point=DocValuesFormat(name=Asserting)}, maxPointsInLeafNode=561, maxMBSortInHeap=5.205976135593324, sim=RandomSimilarity(queryNorm=false,coord=crazy): {}, locale=hr-HR, timezone=UCT
[junit4] 2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=2,free=265220296,total=523239424
[junit4] 2> NOTE: All tests run in this JVM: [TestGeo3DPoint]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestGeo3DPoint -Dtests.seed=1B7B70F64CC3238E -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/test-data/enwiki.random.lines.txt -Dtests.locale=hr-HR -Dtests.timezone=UCT -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J1 | TestGeo3DPoint (suite) <<<
[junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4] > at __randomizedtesting.SeedInfo.seed([1B7B70F64CC3238E]:0)
[junit4] Completed [10/11 (1!)] on J1 in 7225.56s, 4 tests, 2 errors <<< FAILURES!
[...truncated 1899 lines...]
[junit4] Suite: org.apache.solr.cloud.hdfs.HdfsSyncSliceTest
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsSyncSliceTest_7C9675E2CB1A59B5-001/init-core-data-001
[junit4] 2> 804505 WARN (SUITE-HdfsSyncSliceTest-seed#[7C9675E2CB1A59B5]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
[junit4] 2> 804505 INFO (SUITE-HdfsSyncSliceTest-seed#[7C9675E2CB1A59B5]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields
[junit4] 2> 804506 INFO (SUITE-HdfsSyncSliceTest-seed#[7C9675E2CB1A59B5]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 804506 INFO (SUITE-HdfsSyncSliceTest-seed#[7C9675E2CB1A59B5]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 808650 WARN (SUITE-HdfsSyncSliceTest-seed#[7C9675E2CB1A59B5]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 809507 INFO (SUITE-HdfsSyncSliceTest-seed#[7C9675E2CB1A59B5]-worker) [ ] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
[junit4] 2> 809580 WARN (SUITE-HdfsSyncSliceTest-seed#[7C9675E2CB1A59B5]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 809834 INFO (SUITE-HdfsSyncSliceTest-seed#[7C9675E2CB1A59B5]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 810068 INFO (SUITE-HdfsSyncSliceTest-seed#[7C9675E2CB1A59B5]-worker) [ ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/hdfs to ./temp/Jetty_lucene2.us.west_apache_org_40059_hdfs____ec7ygy/webapp
[junit4] 2> 812636 INFO (SUITE-HdfsSyncSliceTest-seed#[7C9675E2CB1A59B5]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@lucene2-us-west.apache.org:40059
[junit4] 2> 817000 WARN (SUITE-HdfsSyncSliceTest-seed#[7C9675E2CB1A59B5]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 817016 INFO (SUITE-HdfsSyncSliceTest-seed#[7C9675E2CB1A59B5]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 817103 INFO (SUITE-HdfsSyncSliceTest-seed#[7C9675E2CB1A59B5]-worker) [ ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_42835_datanode____4axhs3/webapp
[junit4] 2> 818715 INFO (SUITE-HdfsSyncSliceTest-seed#[7C9675E2CB1A59B5]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42835
[junit4] 2> 821511 WARN (SUITE-HdfsSyncSliceTest-seed#[7C9675E2CB1A59B5]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 821512 INFO (SUITE-HdfsSyncSliceTest-seed#[7C9675E2CB1A59B5]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 821618 INFO (SUITE-HdfsSyncSliceTest-seed#[7C9675E2CB1A59B5]-worker) [ ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_41153_datanode____.qqet5z/webapp
[junit4] 2> 823769 INFO (SUITE-HdfsSyncSliceTest-seed#[7C9675E2CB1A59B5]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41153
[junit4] 2> 829675 INFO (IPC Server handler 4 on 41093) [ ] BlockStateChange BLOCK* processReport: from storage DS-868092ff-9c35-4f62-bcc5-30c4c7a98210 node DatanodeRegistration(127.0.0.1:45443, datanodeUuid=d7a91c31-1adc-47fe-b25e-fa239f04c293, infoPort=36959, infoSecurePort=0, ipcPort=33096, storageInfo=lv=-56;cid=testClusterID;nsid=76016721;c=0), blocks: 0, hasStaleStorage: true, processing time: 156 msecs
[junit4] 2> 829683 INFO (IPC Server handler 0 on 41093) [ ] BlockStateChange BLOCK* processReport: from storage DS-5f6daba4-c210-491b-b9cd-aca0da1a8215 node DatanodeRegistration(127.0.0.1:40573, datanodeUuid=d9a74ec0-f947-4ebf-a8e0-dc4d0e083e8e, infoPort=43394, infoSecurePort=0, ipcPort=42848, storageInfo=lv=-56;cid=testClusterID;nsid=76016721;c=0), blocks: 0, hasStaleStorage: true, processing time: 17 msecs
[junit4] 2> 829814 INFO (IPC Server handler 4 on 41093) [ ] BlockStateChange BLOCK* processReport: from storage DS-6825d5f8-8015-4efd-889c-e9a2f1fdf6a3 node DatanodeRegistration(127.0.0.1:45443, datanodeUuid=d7a91c31-1adc-47fe-b25e-fa239f04c293, infoPort=36959, infoSecurePort=0, ipcPort=33096, storageInfo=lv=-56;cid=testClusterID;nsid=76016721;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 829815 INFO (IPC Server handler 0 on 41093) [ ] BlockStateChange BLOCK* processReport: from storage DS-4757d38a-c6e8-4b60-b006-bb84de5d93bf node DatanodeRegistration(127.0.0.1:40573, datanodeUuid=d9a74ec0-f947-4ebf-a8e0-dc4d0e083e8e, infoPort=43394, infoSecurePort=0, ipcPort=42848, storageInfo=lv=-56;cid=testClusterID;nsid=76016721;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 832657 INFO (TEST-HdfsSyncSliceTest.test-seed#[7C9675E2CB1A59B5]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 832676 INFO (Thread-352) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 832676 INFO (Thread-352) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 832758 ERROR (Thread-352) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 832773 INFO (TEST-HdfsSyncSliceTest.test-seed#[7C9675E2CB1A59B5]) [ ] o.a.s.c.ZkTestServer start zk server on port:41414
[junit4] 2> 833137 INFO (TEST-HdfsSyncSliceTest.test-seed#[7C9675E2CB1A59B5]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 833184 INFO (TEST-HdfsSyncSliceTest.test-seed#[7C9675E2CB1A59B5]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 833185 INFO (TEST-HdfsSyncSliceTest.test-seed#[7C9675E2CB1A59B5]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 833188 INFO (TEST-HdfsSyncSliceTest.test-seed#[7C9675E2CB1A59B5]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 833201 INFO (TEST-HdfsSyncSliceTest.test-seed#[7C9675E2CB1A59B5]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 833202 INFO (TEST-HdfsSyncSliceTest.test-seed#[7C9675E2CB1A59B5]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 833203 INFO (TEST-HdfsSyncSliceTest.test-seed#[7C9675E2CB1A59B5]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 833217 INFO (TEST-HdfsSyncSliceTest.test-seed#[7C9675E2CB1A59B5]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 833218 INFO (TEST-HdfsSyncSliceTest.test-seed#[7C9675E2CB1A59B5]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 833219 INFO (TEST-HdfsSyncSliceTest.test-seed#[7C9675E2CB1A59B5]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 833220 INFO (TEST-HdfsSyncSliceTest.test-seed#[7C9675E2CB1A59B5]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 833992 INFO (TEST-HdfsSyncSliceTest.test-seed#[7C9675E2CB1A59B5]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsSyncSliceTest_7C9675E2CB1A59B5-001/control-001/cores/collection1
[junit4] 2> 834073 INFO (TEST-HdfsSyncSliceTest.test-seed#[7C9675E2CB1A59B5]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 834159 INFO (TEST-HdfsSyncSliceTest.test-seed#[7C9675E2CB1A59B5]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4429ef09{/,null,AVAILABLE}
[junit4] 2> 834159 INFO (TEST-HdfsSyncSliceTest.test-seed#[7C9675E2CB1A59B5]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@23800103{HTTP/1.1,[http/1.1]}{127.0.0.1:33605}
[junit4] 2> 834159 INFO (TEST-HdfsSyncSliceTest.test-seed#[7C9675E2CB1A59B5]) [ ] o.e.j.s.Server Started @846106ms
[junit4] 2> 834159 INFO (TEST-HdfsSyncSliceTest.test-seed#[7C9675E2CB1A59B5]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://lucene2-us-west.apache.org:41093/hdfs__lucene2-us-west.apache.org_41093__home_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-6.6_checkout_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.HdfsSyncSliceTest_7C9675E2CB1A59B5-001_tempDir-002_control_data, hostContext=/, hostPort=33605, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsSyncSliceTest_7C9675E2CB1A59B5-001/control-001/cores}
[junit4] 2> 834160 ERROR (TEST-HdfsSyncSliceTest.test-seed#[7C9675E2CB1A59B5]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 834160 INFO (TEST-HdfsSyncSliceTest.test-seed#[7C9675E2CB1A59B5]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.6.1
[junit4] 2> 834160 INFO (TEST-HdfsSyncSliceTest.test-seed#[7C9675E2CB1A59B5]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 834160 INFO (TEST-HdfsSyncSliceTest.test-seed#[7C9675E2CB1A59B5]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 834160 INFO (TEST-HdfsSyncSliceTest.test-seed#[7C9675E2CB1A59B5]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-20T07:16:29.992Z
[junit4] 2> 834261 INFO (TEST-HdfsSyncSliceTest.test-seed#[7C9675E2CB1A59B5]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 834261 INFO (TEST-HdfsSyncSliceTest.test-seed#[7C9675E2CB1A59B5]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsSyncSliceTest_7C9675E2CB1A59B5-001/control-001/solr.xml
[junit4] 2> 834309 INFO (TEST-HdfsSyncSliceTest.test-seed#[7C9675E2CB1A59B5]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 834318 INFO (TEST-HdfsSyncSliceTest.test-seed#[7C9675E2CB1A59B5]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41414/solr
[junit4] 2> 834624 INFO (TEST-HdfsSyncSliceTest.test-seed#[7C9675E2CB1A59B5]) [n:127.0.0.1:33605_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 834625 INFO (TEST-HdfsSyncSliceTest.test-seed#[7C9675E2CB1A59B5]) [n:127.0.0.1:33605_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:33605_
[junit4] 2> 834626 INFO (TEST-HdfsSyncSliceTest.test-seed#[7C9675E2CB1A59B5]) [n:127.0.0.1:33605_ ] o.a.s.c.Overseer Overseer (id=98514592632471556-127.0.0.1:33605_-n_0000000000) starting
[junit4] 2> 834752 INFO (TEST-HdfsSyncSliceTest.test-seed#[7C9675E2CB1A59B5]) [n:127.0.0.1:33605_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33605_
[junit4] 2> 834837 INFO (zkCallback-162-thread-1-processing-n:127.0.0.1:33605_) [n:127.0.0.1:33605_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 835077 INFO (TEST-HdfsSyncSliceTest.test-seed#[7C9675E2CB1A59B5]) [n:127.0.0.1:33605_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsSyncSliceTest_7C9675E2CB1A59B5-001/control-001/cores
[junit4] 2> 835077 INFO (TEST-HdfsSyncSliceTest.test-seed#[7C9675E2CB1A59B5]) [n:127.0.0.1:33605_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 835127 INFO (coreLoadExecutor-443-thread-1-processing-n:127.0.0.1:33605_) [n:127.0.0.1:33605_ c:control_collection x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 835183 INFO (OverseerStateUpdate-98514592632471556-127.0.0.1:33605_-n_0000000000) [n:127.0.0.1:33605_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 836389 INFO (coreLoadExecutor-443-thread-1-processing-n:127.0.0.1:33605_) [n:127.0.0.1:33605_ c:control_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.1
[junit4] 2> 836526 INFO (coreLoadExecutor-443-thread-1-processing-n:127.0.0.1:33605_) [n:127.0.0.1:33605_ c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 837259 INFO (coreLoadExecutor-443-thread-1-processing-n:127.0.0.1:33605_) [n:127.0.0.1:33605_ c:control_collection x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 837533 INFO (coreLoadExecutor-443-thread-1-processing-n:127.0.0.1:33605_) [n:127.0.0.1:33605_ c:control_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection, trusted=true
[junit4] 2> 837582 INFO (coreLoadExecutor-443-thread-1-processing-n:127.0.0.1:33605_) [n:127.0.0.1:33605_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://lucene2-us-west.apache.org:41093/solr_hdfs_home
[junit4] 2> 837582 INFO (coreLoadExecutor-443-thread-1-processing-n:127.0.0.1:33605_) [n:127.0.0.1:33605_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 837582 INFO (coreLoadExecutor-443-thread-1-processing-n:127.0.0.1:33605_) [n:127.0.0.1:33605_ c:control_collection x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 837582 INFO (coreLoadExecutor-443-thread-1-processing-n:127.0.0.1:33605_) [n:127.0.0.1:33605_ c:control_collection x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsSyncSliceTest_7C9675E2CB1A59B5-001/control-001/cores/collection1], dataDir=[hdfs://lucene2-us-west.apache.org:41093/solr_hdfs_home/control_collection/core_node1/data/]
[junit4] 2> 837583 INFO (coreLoadExecutor-443-thread-1-processing-n:127.0.0.1:33605_) [n:127.0.0.1:33605_ c:control_collection x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7fcae36a
[junit4] 2> 837583 INFO (coreLoadExecutor-443-thread-1-processing-n:127.0.0.1:33605_) [n:127.0.0.1:33605_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:41093/solr_hdfs_home/control_collection/core_node1/data/snapshot_metadata
[junit4] 2> 837800 INFO (coreLoadExecutor-443-thread-1-processing-n:127.0.0.1:33605_) [n:127.0.0.1:33605_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 837800 INFO (coreLoadExecutor-443-thread-1-processing-n:127.0.0.1:33605_) [n:127.0.0.1:33605_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 837800 INFO (coreLoadExecutor-443-thread-1-processing-n:127.0.0.1:33605_) [n:127.0.0.1:33605_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 840746 INFO (coreLoadExecutor-443-thread-1-processing-n:127.0.0.1:33605_) [n:127.0.0.1:33605_ c:control_collection x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 840779 INFO (coreLoadExecutor-443-thread-1-processing-n:127.0.0.1:33605_) [n:127.0.0.1:33605_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:41093/solr_hdfs_home/control_collection/core_node1/data
[junit4] 2> 841007 INFO (coreLoadExecutor-443-thread-1-processing-n:127.0.0.1:33605_) [n:127.0.0.1:33605_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:41093/solr_hdfs_home/control_collection/core_node1/data/index
[junit4] 2> 841074 INFO (coreLoadExecutor-443-thread-1-processing-n:127.0.0.1:33605_) [n:127.0.0.1:33605_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 841074 INFO (coreLoadExecutor-443-thread-1-processing-n:127.0.0.1:33605_) [n:127.0.0.1:33605_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 841074 INFO (coreLoadExecutor-443-thread-1-processing-n:127.0.0.1:33605_) [n:127.0.0.1:33605_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 841171 INFO (coreLoadExecutor-443-thread-1-processing-n:127.0.0.1:33605_) [n:127.0.0.1:33605_ c:control_collection x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 841172 INFO (coreLoadExecutor-443-thread-1-processing-n:127.0.0.1:33605_) [n:127.0.0.1:33605_ c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=17, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=92.796875, floorSegmentMB=1.50390625, forceMergeDeletesPctAllowed=21.53293850376789, segmentsPerTier=47.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 842816 INFO (IPC Server handler 5 on 41093) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:40573 is added to blk_1073741825_1001{UCState=COMMITTED, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-6825d5f8-8015-4efd-889c-e9a2f1fdf6a3:NORMAL:127.0.0.1:45443|RBW], ReplicaUC[[DISK]DS-5f6daba4-c210-491b-b9cd-aca0da1a8215:NORMAL:127.0.0.1:40573|RBW]]} size 71
[junit4] 2> 842816 INFO (IPC Server handler 1 on 41093) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:45443 is added to blk_1073741825_1001 size 71
[junit4] 2> 843301 WARN (coreLoadExecutor-443-thread-1-processing-n:127.0.0.1:33605_) [n:127.0.0.1:33605_ c:control_collection x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 843646 INFO (coreLoadExecutor-443-thread-1-processing-n:127.0.0.1:33605_) [n:127.0.0.1:33605_ c:control_collection x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 843646 INFO (coreLoadExecutor-443-thread-1-processing-n:127.0.0.1:33605_) [n:127.0.0.1:33605_ c:control_collection x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 843646 INFO (coreLoadExecutor-443-thread-1-processing-n:127.0.0.1:33605_) [n:127.0.0.1:33605_ c:control_collection x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 843762 INFO (coreLoadExecutor-443-thread-1-processing-n:127.0.0.1:33605_) [n:127.0.0.1:33605_ c:control_collection x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 843762 INFO (coreLoadExecutor-443-thread-1-processing-n:127.0.0.1:33605_) [n:127.0.0.1:33605_ c:control_collection x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 843793 INFO (coreLoadExecutor-443-thread-1-processing-n:127.0.0.1:33605_) [n:127.0.0.1:33605_ c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=26, maxMergeAtOnceExplicit=31, maxMergedSegmentMB=43.357421875, floorSegmentMB=0.5732421875, forceMergeDeletesPctAllowed=1.1922969063030053, segmentsPerTier=16.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6423916812924095
[junit4] 2> 844305 INFO (coreLoadExecutor-443-thread-1-processing-n:127.0.0.1:33605_) [n:127.0.0.1:33605_ c:control_collection x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3b0db439[collection1] main]
[junit4] 2> 844321 INFO (coreLoadExecutor-443-thread-1-processing-n:127.0.0.1:33605_) [n:127.0.0.1:33605_ c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 844321 INFO (coreLoadExecutor-443-thread-1-processing-n:127.0.0.1:33605_) [n:127.0.0.1:33605_ c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 844323 INFO (coreLoadExecutor-443-thread-1-processing-n:127.0.0.1:33605_) [n:127.0.0.1:33605_ c:control_collection x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 844463 INFO (searcherExecutor-444-thread-1-processing-n:127.0.0.1:33605_ x:collection1 c:control_collection) [n:127.0.0.1:33605_ c:control_collection x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@3b0db439[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 844554 INFO (coreLoadExecutor-443-thread-1-processing-n:127.0.0.1:33605_) [n:127.0.0.1:33605_ c:control_collection x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1576233494523150336
[junit4] 2> 844660 INFO (coreZkRegister-436-thread-1-processing-n:127.0.0.1:33605_ x:collection1 c:control_collection) [n:127.0.0.1:33605_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 844660 INFO (coreZkRegister-436-thread-1-processing-n:127.0.0.1:33605_ x:collection1 c:control_collection) [n:127.0.0.1:33605_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 844660 INFO (coreZkRegister-436-thread-1-processing-n:127.0.0.1:33605_ x:collection1 c:control_collection) [n:127.0.0.1:33605_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:33605/collection1/
[junit4] 2> 844660 INFO (coreZkRegister-436-thread-1-processing-n:127.0.0.1:33605_ x:collection1 c:control_collection) [n:127.0.0.1:33605_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 844660 INFO (coreZkRegister-436-thread-1-processing-n:127.0.0.1:33605_ x:collection1 c:control_collection) [n:127.0.0.1:33605_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:33605/collection1/ has no replicas
[junit4] 2> 844660 INFO (coreZkRegister-436-thread-1-processing-n:127.0.0.1:33605_ x:collection1 c:control_collection) [n:127.0.0.1:33605_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 844681 INFO (coreZkRegister-436-thread-1-processing-n:127.0.0.1:33605_ x:collection1 c:control_collection) [n:127.0.0.1:33605_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:33605/collection1/ shard1
[junit4] 2> 844842 INFO (coreZkRegister-436-thread-1-processing-n:127.0.0.1:33605_ x:collection1 c:control_collection) [n:127.0.0.1:33605_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 845207 INFO (TEST-HdfsSyncSliceTest.test-seed#[7C9675E2CB1A59B5]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 845223 INFO (TEST-HdfsSyncSliceTest.test-seed#[7C9675E2CB1A59B5]) [ ] o.a.s.c.s.i.ZkClientCluste
[...truncated too long message...]
ca1] o.a.s.c.SolrCore [testcollection_shard4_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@528f68ec
[junit4] 2> 616523 INFO (zkCallback-1706-thread-2-processing-n:127.0.0.1:44406_solr) [n:127.0.0.1:44406_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 616524 INFO (zkCallback-1703-thread-2-processing-n:127.0.0.1:45473_solr) [n:127.0.0.1:45473_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 616524 INFO (zkCallback-1722-thread-2-processing-n:127.0.0.1:46731_solr) [n:127.0.0.1:46731_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 616564 INFO (coreCloseExecutor-814-thread-1) [n:127.0.0.1:45473_solr c:falsepredicate s:shard2 r:core_node4 x:falsepredicate_shard2_replica1] o.a.s.c.SolrCore [falsepredicate_shard2_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@67362512
[junit4] 2> 616564 INFO (coreCloseExecutor-814-thread-2) [n:127.0.0.1:45473_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica1] o.a.s.c.SolrCore [testcollection_shard1_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@7b51eeb0
[junit4] 2> 616565 INFO (coreCloseExecutor-815-thread-1) [n:127.0.0.1:46731_solr c:falsepredicate s:shard3 r:core_node5 x:falsepredicate_shard3_replica1] o.a.s.c.SolrCore [falsepredicate_shard3_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@7ce8f7c0
[junit4] 2> 616565 INFO (coreCloseExecutor-815-thread-2) [n:127.0.0.1:46731_solr c:testcollection s:shard3 r:core_node4 x:testcollection_shard3_replica1] o.a.s.c.SolrCore [testcollection_shard3_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@2a85d790
[junit4] 2> 616565 INFO (coreCloseExecutor-815-thread-3) [n:127.0.0.1:46731_solr c:stateformat1 s:shard1 r:core_node1 x:stateformat1_shard1_replica1] o.a.s.c.SolrCore [stateformat1_shard1_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@500d3d99
[junit4] 2> 616566 INFO (coreCloseExecutor-814-thread-3) [n:127.0.0.1:45473_solr c:waitforstate s:shard1 r:core_node1 x:waitforstate_shard1_replica1] o.a.s.c.SolrCore [waitforstate_shard1_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@1782ca79
[junit4] 2> 616574 INFO (coreCloseExecutor-813-thread-1) [n:127.0.0.1:44406_solr c:falsepredicate s:shard4 r:core_node3 x:falsepredicate_shard4_replica1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.falsepredicate.shard4.replica1
[junit4] 2> 616577 INFO (coreCloseExecutor-814-thread-1) [n:127.0.0.1:45473_solr c:falsepredicate s:shard2 r:core_node4 x:falsepredicate_shard2_replica1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.falsepredicate.shard2.replica1
[junit4] 2> 616587 INFO (coreCloseExecutor-813-thread-2) [n:127.0.0.1:44406_solr c:testcollection s:shard4 r:core_node1 x:testcollection_shard4_replica1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.testcollection.shard4.replica1
[junit4] 2> 616623 INFO (coreCloseExecutor-815-thread-2) [n:127.0.0.1:46731_solr c:testcollection s:shard3 r:core_node4 x:testcollection_shard3_replica1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.testcollection.shard3.replica1
[junit4] 2> 616631 INFO (coreCloseExecutor-815-thread-1) [n:127.0.0.1:46731_solr c:falsepredicate s:shard3 r:core_node5 x:falsepredicate_shard3_replica1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.falsepredicate.shard3.replica1
[junit4] 2> 616632 INFO (coreCloseExecutor-814-thread-2) [n:127.0.0.1:45473_solr c:testcollection s:shard1 r:core_node3 x:testcollection_shard1_replica1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.testcollection.shard1.replica1
[junit4] 2> 616633 INFO (coreCloseExecutor-814-thread-3) [n:127.0.0.1:45473_solr c:waitforstate s:shard1 r:core_node1 x:waitforstate_shard1_replica1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.waitforstate.shard1.replica1
[junit4] 2> 616633 WARN (zkCallback-1706-thread-2-processing-n:127.0.0.1:44406_solr) [n:127.0.0.1:44406_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 616634 INFO (jetty-closer-1685-thread-1) [ ] o.a.s.c.Overseer Overseer (id=98515229629677577-127.0.0.1:45473_solr-n_0000000000) closing
[junit4] 2> 616634 INFO (jetty-closer-1685-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2864f9e4{/solr,null,UNAVAILABLE}
[junit4] 2> 616634 INFO (zkCallback-1703-thread-2-processing-n:127.0.0.1:45473_solr) [n:127.0.0.1:45473_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 616634 INFO (OverseerStateUpdate-98515229629677577-127.0.0.1:45473_solr-n_0000000000) [n:127.0.0.1:45473_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:45473_solr
[junit4] 2> 616634 INFO (zkCallback-1722-thread-1-processing-n:127.0.0.1:46731_solr) [n:127.0.0.1:46731_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 616664 WARN (zkCallback-1703-thread-2-processing-n:127.0.0.1:45473_solr) [n:127.0.0.1:45473_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 616665 INFO (zkCallback-1722-thread-2-processing-n:127.0.0.1:46731_solr) [n:127.0.0.1:46731_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:46731_solr
[junit4] 2> 616665 INFO (zkCallback-1722-thread-1-processing-n:127.0.0.1:46731_solr) [n:127.0.0.1:46731_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 616670 INFO (coreCloseExecutor-815-thread-3) [n:127.0.0.1:46731_solr c:stateformat1 s:shard1 r:core_node1 x:stateformat1_shard1_replica1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.stateformat1.shard1.replica1
[junit4] 2> 616670 INFO (jetty-closer-1685-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@152f1678{/solr,null,UNAVAILABLE}
[junit4] 2> 618185 WARN (zkCallback-1722-thread-2-processing-n:127.0.0.1:46731_solr) [n:127.0.0.1:46731_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 618185 INFO (jetty-closer-1685-thread-3) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@43de4979{/solr,null,UNAVAILABLE}
[junit4] 2> 618186 ERROR (SUITE-TestCollectionStateWatchers-seed#[EAD0DD62EFD59022]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 618186 INFO (SUITE-TestCollectionStateWatchers-seed#[EAD0DD62EFD59022]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:38970 38970
[junit4] 2> 618487 INFO (Thread-443) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:38970 38970
[junit4] 2> 618488 WARN (Thread-443) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 8 /solr/aliases.json
[junit4] 2> 7 /solr/security.json
[junit4] 2> 6 /solr/configs/config
[junit4] 2> 3 /solr/collections/stateformat1/state.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 12 /solr/collections/waitforstate/state.json
[junit4] 2> 8 /solr/clusterprops.json
[junit4] 2> 8 /solr/clusterstate.json
[junit4] 2> 7 /solr/collections/falsepredicate/state.json
[junit4] 2> 7 /solr/collections/testcollection/state.json
[junit4] 2> 3 /solr/overseer_elect/election/98515229629677577-127.0.0.1:45473_solr-n_0000000000
[junit4] 2> 3 /solr/collections/stateformat1/state.json
[junit4] 2> 2 /solr/collections/currentstate/state.json
[junit4] 2> 2 /solr/overseer_elect/election/98515229629677588-127.0.0.1:46731_solr-n_0000000004
[junit4] 2> 2 /solr/collections/delayed/state.json
[junit4] 2> 2 /solr/collections/tobedeleted/state.json
[junit4] 2> 2 /solr/overseer_elect/election/98515229629677580-127.0.0.1:44406_solr-n_0000000002
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 8 /solr/collections
[junit4] 2> 5 /solr/live_nodes
[junit4] 2>
[junit4] 2> 618488 INFO (SUITE-TestCollectionStateWatchers-seed#[EAD0DD62EFD59022]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-solrj/test/J2/temp/solr.common.cloud.TestCollectionStateWatchers_EAD0DD62EFD59022-001
[junit4] 2> Aug 20, 2017 9:59:38 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): {}, docValues:{}, maxPointsInLeafNode=624, maxMBSortInHeap=5.094515933401851, sim=RandomSimilarity(queryNorm=true,coord=no): {}, locale=es-PA, timezone=Europe/Lisbon
[junit4] 2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=41741096,total=402128896
[junit4] 2> NOTE: All tests run in this JVM: [SchemaTest, LargeVolumeJettyTest, TangentEvaluatorTest, SolrParamTest, SolrExampleBinaryTest, TestHash, ModifiableSolrParamsTest, StreamExpressionTest, TestJavaBinCodec, TestCollectionStateWatchers]
[junit4] Completed [117/130 (1!)] on J2 in 70.99s, 9 tests, 1 failure <<< FAILURES!
[...truncated 42683 lines...]
[JENKINS] Lucene-Solr-NightlyTests-6.6 - Build # 38 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.6/38/
3 tests failed.
FAILED: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testCollectionsAPI
Error Message:
Something is broken in the assert for no shards using the same indexDir - probably something was changed in the attributes published in the MBean of SolrCore : {}
Stack Trace:
java.lang.AssertionError: Something is broken in the assert for no shards using the same indexDir - probably something was changed in the attributes published in the MBean of SolrCore : {}
at __randomizedtesting.SeedInfo.seed([5FBDCDA8573442E0:17C8B91C51076D75]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.checkNoTwoShardsUseTheSameIndexDir(CollectionsAPIDistributedZkTest.java:646)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:524)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.hdfs.StressHdfsTest.test
Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:47796/_pto
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:47796/_pto
at __randomizedtesting.SeedInfo.seed([5FBDCDA8573442E0:D7E9F272F9C82F18]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:623)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:447)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:388)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1383)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1134)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1073)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at org.apache.solr.cloud.hdfs.StressHdfsTest.createAndDeleteCollection(StressHdfsTest.java:220)
at org.apache.solr.cloud.hdfs.StressHdfsTest.test(StressHdfsTest.java:103)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:515)
... 51 more
FAILED: org.apache.solr.cloud.hdfs.HdfsUnloadDistributedZkTest.test
Error Message:
Captured an uncaught exception in thread: Thread[id=15220, name=testExecutor-2507-thread-10, state=RUNNABLE, group=TGRP-HdfsUnloadDistributedZkTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=15220, name=testExecutor-2507-thread-10, state=RUNNABLE, group=TGRP-HdfsUnloadDistributedZkTest]
at __randomizedtesting.SeedInfo.seed([5FBDCDA8573442E0:D7E9F272F9C82F18]:0)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:51812: Error CREATEing SolrCore 'multiunload9': Unable to create core [multiunload9] Caused by: Direct buffer memory
at __randomizedtesting.SeedInfo.seed([5FBDCDA8573442E0]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:612)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at org.apache.solr.cloud.BasicDistributedZkTest.lambda$createCollectionInOneInstance$0(BasicDistributedZkTest.java:589)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12464 lines...]
[junit4] Suite: org.apache.solr.cloud.hdfs.HdfsUnloadDistributedZkTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_5FBDCDA8573442E0-001/init-core-data-001
[junit4] 2> 840095 WARN (SUITE-HdfsUnloadDistributedZkTest-seed#[5FBDCDA8573442E0]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=4 numCloses=4
[junit4] 2> 840096 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[5FBDCDA8573442E0]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields
[junit4] 2> 840097 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[5FBDCDA8573442E0]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
[junit4] 2> 840097 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[5FBDCDA8573442E0]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 840148 WARN (SUITE-HdfsUnloadDistributedZkTest-seed#[5FBDCDA8573442E0]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 840174 WARN (SUITE-HdfsUnloadDistributedZkTest-seed#[5FBDCDA8573442E0]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 840181 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[5FBDCDA8573442E0]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 840206 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[5FBDCDA8573442E0]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_60296_hdfs____.k3gbk1/webapp
[junit4] 2> 840665 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[5FBDCDA8573442E0]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:60296
[junit4] 2> 840775 WARN (SUITE-HdfsUnloadDistributedZkTest-seed#[5FBDCDA8573442E0]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 840776 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[5FBDCDA8573442E0]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 840795 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[5FBDCDA8573442E0]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_59666_datanode____tudoop/webapp
[junit4] 2> 841272 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[5FBDCDA8573442E0]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:59666
[junit4] 2> 841349 WARN (SUITE-HdfsUnloadDistributedZkTest-seed#[5FBDCDA8573442E0]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 841351 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[5FBDCDA8573442E0]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 841365 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[5FBDCDA8573442E0]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_37376_datanode____.a5pjmz/webapp
[junit4] 2> 841834 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[5FBDCDA8573442E0]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37376
[junit4] 2> 841979 ERROR (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_5FBDCDA8573442E0-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_5FBDCDA8573442E0-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to localhost/127.0.0.1:50475) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 841993 ERROR (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_5FBDCDA8573442E0-001/tempDir-001/hdfsBaseDir/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_5FBDCDA8573442E0-001/tempDir-001/hdfsBaseDir/data/data4/]] heartbeating to localhost/127.0.0.1:50475) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 842007 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x49c75e8d85a701: from storage DS-1c9d5cb1-5254-4efe-bd60-28d2595886bb node DatanodeRegistration(127.0.0.1:43167, datanodeUuid=81bcc8e7-3d01-448a-9d83-8aa88358d350, infoPort=43749, infoSecurePort=0, ipcPort=47142, storageInfo=lv=-56;cid=testClusterID;nsid=746833506;c=0), blocks: 0, hasStaleStorage: true, processing time: 1 msecs
[junit4] 2> 842007 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x49c75e8d85f96d: from storage DS-b59e5c3f-1dff-4764-9bf2-322b33d05f46 node DatanodeRegistration(127.0.0.1:36367, datanodeUuid=380b4549-d68f-4b69-85ec-418c677229eb, infoPort=38437, infoSecurePort=0, ipcPort=55971, storageInfo=lv=-56;cid=testClusterID;nsid=746833506;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 842007 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x49c75e8d85a701: from storage DS-f41f3bb6-b5cd-4078-8203-8b333fac5fc3 node DatanodeRegistration(127.0.0.1:43167, datanodeUuid=81bcc8e7-3d01-448a-9d83-8aa88358d350, infoPort=43749, infoSecurePort=0, ipcPort=47142, storageInfo=lv=-56;cid=testClusterID;nsid=746833506;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 842007 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x49c75e8d85f96d: from storage DS-41fb63d0-80d6-461e-8463-0a2d5b3f9fb0 node DatanodeRegistration(127.0.0.1:36367, datanodeUuid=380b4549-d68f-4b69-85ec-418c677229eb, infoPort=38437, infoSecurePort=0, ipcPort=55971, storageInfo=lv=-56;cid=testClusterID;nsid=746833506;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 842480 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 842493 INFO (Thread-9244) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 842493 INFO (Thread-9244) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 842503 ERROR (Thread-9244) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 842593 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.c.ZkTestServer start zk server on port:35511
[junit4] 2> 842606 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 842608 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 842610 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 842611 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 842613 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 842614 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 842616 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 842619 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 842620 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 842621 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 842623 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 842739 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_5FBDCDA8573442E0-001/control-001/cores/collection1
[junit4] 2> 842741 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 842742 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4e2593b0{/,null,AVAILABLE}
[junit4] 2> 842743 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@51f17164{HTTP/1.1,[http/1.1]}{127.0.0.1:46394}
[junit4] 2> 842747 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.e.j.s.Server Started @845876ms
[junit4] 2> 842747 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:50475/hdfs__localhost_50475__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-6.6_checkout_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsUnloadDistributedZkTest_5FBDCDA8573442E0-001_tempDir-002_control_data, hostContext=/, hostPort=46394, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_5FBDCDA8573442E0-001/control-001/cores}
[junit4] 2> 842748 ERROR (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 842748 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.6.4
[junit4] 2> 842748 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 842748 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 842748 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-03-08T13:16:20.360Z
[junit4] 2> 842752 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 842752 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_5FBDCDA8573442E0-001/control-001/solr.xml
[junit4] 2> 842763 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 842765 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35511/solr
[junit4] 2> 842794 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [n:127.0.0.1:46394_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 842794 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [n:127.0.0.1:46394_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:46394_
[junit4] 2> 842795 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [n:127.0.0.1:46394_ ] o.a.s.c.Overseer Overseer (id=99648469736620036-127.0.0.1:46394_-n_0000000000) starting
[junit4] 2> 842806 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [n:127.0.0.1:46394_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46394_
[junit4] 2> 842813 INFO (zkCallback-1285-thread-1-processing-n:127.0.0.1:46394_) [n:127.0.0.1:46394_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 842893 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [n:127.0.0.1:46394_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_5FBDCDA8573442E0-001/control-001/cores
[junit4] 2> 842893 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [n:127.0.0.1:46394_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 842895 INFO (coreLoadExecutor-2429-thread-1-processing-n:127.0.0.1:46394_) [n:127.0.0.1:46394_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 842905 INFO (OverseerStateUpdate-99648469736620036-127.0.0.1:46394_-n_0000000000) [n:127.0.0.1:46394_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 843921 INFO (coreLoadExecutor-2429-thread-1-processing-n:127.0.0.1:46394_) [n:127.0.0.1:46394_ c:control_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.4
[junit4] 2> 843955 INFO (coreLoadExecutor-2429-thread-1-processing-n:127.0.0.1:46394_) [n:127.0.0.1:46394_ c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 844106 INFO (coreLoadExecutor-2429-thread-1-processing-n:127.0.0.1:46394_) [n:127.0.0.1:46394_ c:control_collection x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 844123 INFO (coreLoadExecutor-2429-thread-1-processing-n:127.0.0.1:46394_) [n:127.0.0.1:46394_ c:control_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection, trusted=true
[junit4] 2> 844123 INFO (coreLoadExecutor-2429-thread-1-processing-n:127.0.0.1:46394_) [n:127.0.0.1:46394_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:50475/solr_hdfs_home
[junit4] 2> 844123 INFO (coreLoadExecutor-2429-thread-1-processing-n:127.0.0.1:46394_) [n:127.0.0.1:46394_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 844123 INFO (coreLoadExecutor-2429-thread-1-processing-n:127.0.0.1:46394_) [n:127.0.0.1:46394_ c:control_collection x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 844123 INFO (coreLoadExecutor-2429-thread-1-processing-n:127.0.0.1:46394_) [n:127.0.0.1:46394_ c:control_collection x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_5FBDCDA8573442E0-001/control-001/cores/collection1], dataDir=[hdfs://localhost:50475/solr_hdfs_home/control_collection/core_node1/data/]
[junit4] 2> 844124 INFO (coreLoadExecutor-2429-thread-1-processing-n:127.0.0.1:46394_) [n:127.0.0.1:46394_ c:control_collection x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@33d2f39d
[junit4] 2> 844124 INFO (coreLoadExecutor-2429-thread-1-processing-n:127.0.0.1:46394_) [n:127.0.0.1:46394_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:50475/solr_hdfs_home/control_collection/core_node1/data/snapshot_metadata
[junit4] 2> 844135 INFO (coreLoadExecutor-2429-thread-1-processing-n:127.0.0.1:46394_) [n:127.0.0.1:46394_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 844135 INFO (coreLoadExecutor-2429-thread-1-processing-n:127.0.0.1:46394_) [n:127.0.0.1:46394_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 844135 INFO (coreLoadExecutor-2429-thread-1-processing-n:127.0.0.1:46394_) [n:127.0.0.1:46394_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 844144 INFO (coreLoadExecutor-2429-thread-1-processing-n:127.0.0.1:46394_) [n:127.0.0.1:46394_ c:control_collection x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 844145 INFO (coreLoadExecutor-2429-thread-1-processing-n:127.0.0.1:46394_) [n:127.0.0.1:46394_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:50475/solr_hdfs_home/control_collection/core_node1/data
[junit4] 2> 844165 INFO (coreLoadExecutor-2429-thread-1-processing-n:127.0.0.1:46394_) [n:127.0.0.1:46394_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:50475/solr_hdfs_home/control_collection/core_node1/data/index
[junit4] 2> 844172 INFO (coreLoadExecutor-2429-thread-1-processing-n:127.0.0.1:46394_) [n:127.0.0.1:46394_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 844172 INFO (coreLoadExecutor-2429-thread-1-processing-n:127.0.0.1:46394_) [n:127.0.0.1:46394_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 844172 INFO (coreLoadExecutor-2429-thread-1-processing-n:127.0.0.1:46394_) [n:127.0.0.1:46394_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 844178 INFO (coreLoadExecutor-2429-thread-1-processing-n:127.0.0.1:46394_) [n:127.0.0.1:46394_ c:control_collection x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 844178 INFO (coreLoadExecutor-2429-thread-1-processing-n:127.0.0.1:46394_) [n:127.0.0.1:46394_ c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=26, maxMergeAtOnceExplicit=45, maxMergedSegmentMB=29.3125, floorSegmentMB=0.25390625, forceMergeDeletesPctAllowed=0.9764929896714247, segmentsPerTier=50.0, maxCFSSegmentSizeMB=1.5185546875, noCFSRatio=0.30202736206383585
[junit4] 2> 844194 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:36367 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-f41f3bb6-b5cd-4078-8203-8b333fac5fc3:NORMAL:127.0.0.1:43167|RBW], ReplicaUC[[DISK]DS-41fb63d0-80d6-461e-8463-0a2d5b3f9fb0:NORMAL:127.0.0.1:36367|FINALIZED]]} size 0
[junit4] 2> 844196 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:43167 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-f41f3bb6-b5cd-4078-8203-8b333fac5fc3:NORMAL:127.0.0.1:43167|RBW], ReplicaUC[[DISK]DS-41fb63d0-80d6-461e-8463-0a2d5b3f9fb0:NORMAL:127.0.0.1:36367|FINALIZED]]} size 0
[junit4] 2> 844210 WARN (coreLoadExecutor-2429-thread-1-processing-n:127.0.0.1:46394_) [n:127.0.0.1:46394_ c:control_collection x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 844254 INFO (coreLoadExecutor-2429-thread-1-processing-n:127.0.0.1:46394_) [n:127.0.0.1:46394_ c:control_collection x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 844254 INFO (coreLoadExecutor-2429-thread-1-processing-n:127.0.0.1:46394_) [n:127.0.0.1:46394_ c:control_collection x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 844254 INFO (coreLoadExecutor-2429-thread-1-processing-n:127.0.0.1:46394_) [n:127.0.0.1:46394_ c:control_collection x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 844274 INFO (coreLoadExecutor-2429-thread-1-processing-n:127.0.0.1:46394_) [n:127.0.0.1:46394_ c:control_collection x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 844274 INFO (coreLoadExecutor-2429-thread-1-processing-n:127.0.0.1:46394_) [n:127.0.0.1:46394_ c:control_collection x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 844280 INFO (coreLoadExecutor-2429-thread-1-processing-n:127.0.0.1:46394_) [n:127.0.0.1:46394_ c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=7, maxMergeAtOnceExplicit=6, maxMergedSegmentMB=32.333984375, floorSegmentMB=1.826171875, forceMergeDeletesPctAllowed=21.043811099767726, segmentsPerTier=14.0, maxCFSSegmentSizeMB=0.912109375, noCFSRatio=1.0
[junit4] 2> 844290 INFO (coreLoadExecutor-2429-thread-1-processing-n:127.0.0.1:46394_) [n:127.0.0.1:46394_ c:control_collection x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@584668d9[collection1] main]
[junit4] 2> 844292 INFO (coreLoadExecutor-2429-thread-1-processing-n:127.0.0.1:46394_) [n:127.0.0.1:46394_ c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 844292 INFO (coreLoadExecutor-2429-thread-1-processing-n:127.0.0.1:46394_) [n:127.0.0.1:46394_ c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 844292 INFO (coreLoadExecutor-2429-thread-1-processing-n:127.0.0.1:46394_) [n:127.0.0.1:46394_ c:control_collection x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 844295 INFO (searcherExecutor-2430-thread-1-processing-n:127.0.0.1:46394_ x:collection1 c:control_collection) [n:127.0.0.1:46394_ c:control_collection x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@584668d9[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 844300 INFO (coreLoadExecutor-2429-thread-1-processing-n:127.0.0.1:46394_) [n:127.0.0.1:46394_ c:control_collection x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1594375517673357312
[junit4] 2> 844307 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 844308 INFO (coreZkRegister-2422-thread-1-processing-n:127.0.0.1:46394_ x:collection1 c:control_collection) [n:127.0.0.1:46394_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 844308 INFO (coreZkRegister-2422-thread-1-processing-n:127.0.0.1:46394_ x:collection1 c:control_collection) [n:127.0.0.1:46394_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 844308 INFO (coreZkRegister-2422-thread-1-processing-n:127.0.0.1:46394_ x:collection1 c:control_collection) [n:127.0.0.1:46394_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:46394/collection1/
[junit4] 2> 844308 INFO (coreZkRegister-2422-thread-1-processing-n:127.0.0.1:46394_ x:collection1 c:control_collection) [n:127.0.0.1:46394_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 844308 INFO (coreZkRegister-2422-thread-1-processing-n:127.0.0.1:46394_ x:collection1 c:control_collection) [n:127.0.0.1:46394_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:46394/collection1/ has no replicas
[junit4] 2> 844308 INFO (coreZkRegister-2422-thread-1-processing-n:127.0.0.1:46394_ x:collection1 c:control_collection) [n:127.0.0.1:46394_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 844309 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:35511/solr ready
[junit4] 2> 844309 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 844309 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
[junit4] 2> 844314 INFO (coreZkRegister-2422-thread-1-processing-n:127.0.0.1:46394_ x:collection1 c:control_collection) [n:127.0.0.1:46394_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:46394/collection1/ shard1
[junit4] 2> 844428 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_5FBDCDA8573442E0-001/shard-1-001/cores/collection1
[junit4] 2> 844429 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_5FBDCDA8573442E0-001/shard-1-001
[junit4] 2> 844429 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 844431 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7500575f{/,null,AVAILABLE}
[junit4] 2> 844431 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2bc1ce0e{HTTP/1.1,[http/1.1]}{127.0.0.1:51812}
[junit4] 2> 844432 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.e.j.s.Server Started @847560ms
[junit4] 2> 844432 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:50475/hdfs__localhost_50475__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-6.6_checkout_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsUnloadDistributedZkTest_5FBDCDA8573442E0-001_tempDir-002_jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=51812, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_5FBDCDA8573442E0-001/shard-1-001/cores}
[junit4] 2> 844432 ERROR (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 844433 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.6.4
[junit4] 2> 844433 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 844433 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 844433 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-03-08T13:16:22.045Z
[junit4] 2> 844436 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 844436 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_5FBDCDA8573442E0-001/shard-1-001/solr.xml
[junit4] 2> 844443 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 844443 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35511/solr
[junit4] 2> 844455 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [n:127.0.0.1:51812_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 844457 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [n:127.0.0.1:51812_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 844460 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [n:127.0.0.1:51812_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51812_
[junit4] 2> 844461 INFO (zkCallback-1289-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 844463 INFO (zkCallback-1285-thread-1-processing-n:127.0.0.1:46394_) [n:127.0.0.1:46394_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 844463 INFO (zkCallback-1295-thread-1-processing-n:127.0.0.1:51812_) [n:127.0.0.1:51812_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 844465 INFO (coreZkRegister-2422-thread-1-processing-n:127.0.0.1:46394_ x:collection1 c:control_collection) [n:127.0.0.1:46394_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 844548 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [n:127.0.0.1:51812_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_5FBDCDA8573442E0-001/shard-1-001/cores
[junit4] 2> 844548 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [n:127.0.0.1:51812_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 844549 INFO (coreLoadExecutor-2440-thread-1-processing-n:127.0.0.1:51812_) [n:127.0.0.1:51812_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 844551 INFO (OverseerStateUpdate-99648469736620036-127.0.0.1:46394_-n_0000000000) [n:127.0.0.1:46394_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 844655 INFO (zkCallback-1295-thread-1-processing-n:127.0.0.1:51812_) [n:127.0.0.1:51812_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 845567 INFO (coreLoadExecutor-2440-thread-1-processing-n:127.0.0.1:51812_) [n:127.0.0.1:51812_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.4
[junit4] 2> 845582 INFO (coreLoadExecutor-2440-thread-1-processing-n:127.0.0.1:51812_) [n:127.0.0.1:51812_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 845733 INFO (coreLoadExecutor-2440-thread-1-processing-n:127.0.0.1:51812_) [n:127.0.0.1:51812_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 845746 INFO (coreLoadExecutor-2440-thread-1-processing-n:127.0.0.1:51812_) [n:127.0.0.1:51812_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1, trusted=true
[junit4] 2> 845747 INFO (coreLoadExecutor-2440-thread-1-processing-n:127.0.0.1:51812_) [n:127.0.0.1:51812_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:50475/solr_hdfs_home
[junit4] 2> 845747 INFO (coreLoadExecutor-2440-thread-1-processing-n:127.0.0.1:51812_) [n:127.0.0.1:51812_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 845747 INFO (coreLoadExecutor-2440-thread-1-processing-n:127.0.0.1:51812_) [n:127.0.0.1:51812_ c:collection1 x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 845747 INFO (coreLoadExecutor-2440-thread-1-processing-n:127.0.0.1:51812_) [n:127.0.0.1:51812_ c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_5FBDCDA8573442E0-001/shard-1-001/cores/collection1], dataDir=[hdfs://localhost:50475/solr_hdfs_home/collection1/core_node1/data/]
[junit4] 2> 845747 INFO (coreLoadExecutor-2440-thread-1-processing-n:127.0.0.1:51812_) [n:127.0.0.1:51812_ c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@33d2f39d
[junit4] 2> 845747 INFO (coreLoadExecutor-2440-thread-1-processing-n:127.0.0.1:51812_) [n:127.0.0.1:51812_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:50475/solr_hdfs_home/collection1/core_node1/data/snapshot_metadata
[junit4] 2> 845756 INFO (coreLoadExecutor-2440-thread-1-processing-n:127.0.0.1:51812_) [n:127.0.0.1:51812_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 845756 INFO (coreLoadExecutor-2440-thread-1-processing-n:127.0.0.1:51812_) [n:127.0.0.1:51812_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 845756 INFO (coreLoadExecutor-2440-thread-1-processing-n:127.0.0.1:51812_) [n:127.0.0.1:51812_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 845763 INFO (coreLoadExecutor-2440-thread-1-processing-n:127.0.0.1:51812_) [n:127.0.0.1:51812_ c:collection1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 845764 INFO (coreLoadExecutor-2440-thread-1-processing-n:127.0.0.1:51812_) [n:127.0.0.1:51812_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:50475/solr_hdfs_home/collection1/core_node1/data
[junit4] 2> 845782 INFO (coreLoadExecutor-2440-thread-1-processing-n:127.0.0.1:51812_) [n:127.0.0.1:51812_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:50475/solr_hdfs_home/collection1/core_node1/data/index
[junit4] 2> 845789 INFO (coreLoadExecutor-2440-thread-1-processing-n:127.0.0.1:51812_) [n:127.0.0.1:51812_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 845789 INFO (coreLoadExecutor-2440-thread-1-processing-n:127.0.0.1:51812_) [n:127.0.0.1:51812_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 845789 INFO (coreLoadExecutor-2440-thread-1-processing-n:127.0.0.1:51812_) [n:127.0.0.1:51812_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 845796 INFO (coreLoadExecutor-2440-thread-1-processing-n:127.0.0.1:51812_) [n:127.0.0.1:51812_ c:collection1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 845797 INFO (coreLoadExecutor-2440-thread-1-processing-n:127.0.0.1:51812_) [n:127.0.0.1:51812_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=26, maxMergeAtOnceExplicit=45, maxMergedSegmentMB=29.3125, floorSegmentMB=0.25390625, forceMergeDeletesPctAllowed=0.9764929896714247, segmentsPerTier=50.0, maxCFSSegmentSizeMB=1.5185546875, noCFSRatio=0.30202736206383585
[junit4] 2> 845816 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:36367 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-1c9d5cb1-5254-4efe-bd60-28d2595886bb:NORMAL:127.0.0.1:43167|RBW], ReplicaUC[[DISK]DS-b59e5c3f-1dff-4764-9bf2-322b33d05f46:NORMAL:127.0.0.1:36367|RBW]]} size 0
[junit4] 2> 845819 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:43167 is added to blk_1073741826_1002 size 71
[junit4] 2> 845833 WARN (coreLoadExecutor-2440-thread-1-processing-n:127.0.0.1:51812_) [n:127.0.0.1:51812_ c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 845887 INFO (coreLoadExecutor-2440-thread-1-processing-n:127.0.0.1:51812_) [n:127.0.0.1:51812_ c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 845887 INFO (coreLoadExecutor-2440-thread-1-processing-n:127.0.0.1:51812_) [n:127.0.0.1:51812_ c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 845887 INFO (coreLoadExecutor-2440-thread-1-processing-n:127.0.0.1:51812_) [n:127.0.0.1:51812_ c:collection1 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 845901 INFO (coreLoadExecutor-2440-thread-1-processing-n:127.0.0.1:51812_) [n:127.0.0.1:51812_ c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 845901 INFO (coreLoadExecutor-2440-thread-1-processing-n:127.0.0.1:51812_) [n:127.0.0.1:51812_ c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 845903 INFO (coreLoadExecutor-2440-thread-1-processing-n:127.0.0.1:51812_) [n:127.0.0.1:51812_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=7, maxMergeAtOnceExplicit=6, maxMergedSegmentMB=32.333984375, floorSegmentMB=1.826171875, forceMergeDeletesPctAllowed=21.043811099767726, segmentsPerTier=14.0, maxCFSSegmentSizeMB=0.912109375, noCFSRatio=1.0
[junit4] 2> 845908 INFO (coreLoadExecutor-2440-thread-1-processing-n:127.0.0.1:51812_) [n:127.0.0.1:51812_ c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@7f150de[collection1] main]
[junit4] 2> 845909 INFO (coreLoadExecutor-2440-thread-1-processing-n:127.0.0.1:51812_) [n:127.0.0.1:51812_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 845910 INFO (coreLoadExecutor-2440-thread-1-processing-n:127.0.0.1:51812_) [n:127.0.0.1:51812_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 845910 INFO (coreLoadExecutor-2440-thread-1-processing-n:127.0.0.1:51812_) [n:127.0.0.1:51812_ c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 845923 INFO (searcherExecutor-2441-thread-1-processing-n:127.0.0.1:51812_ x:collection1 c:collection1) [n:127.0.0.1:51812_ c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@7f150de[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 845926 INFO (coreLoadExecutor-2440-thread-1-processing-n:127.0.0.1:51812_) [n:127.0.0.1:51812_ c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1594375519378341888
[junit4] 2> 845942 INFO (coreZkRegister-2435-thread-1-processing-n:127.0.0.1:51812_ x:collection1 c:collection1) [n:127.0.0.1:51812_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 845942 INFO (coreZkRegister-2435-thread-1-processing-n:127.0.0.1:51812_ x:collection1 c:collection1) [n:127.0.0.1:51812_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 845942 INFO (coreZkRegister-2435-thread-1-processing-n:127.0.0.1:51812_ x:collection1 c:collection1) [n:127.0.0.1:51812_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:51812/collection1/
[junit4] 2> 845942 INFO (coreZkRegister-2435-thread-1-processing-n:127.0.0.1:51812_ x:collection1 c:collection1) [n:127.0.0.1:51812_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 845942 INFO (coreZkRegister-2435-thread-1-processing-n:127.0.0.1:51812_ x:collection1 c:collection1) [n:127.0.0.1:51812_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:51812/collection1/ has no replicas
[junit4] 2> 845943 INFO (coreZkRegister-2435-thread-1-processing-n:127.0.0.1:51812_ x:collection1 c:collection1) [n:127.0.0.1:51812_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 845957 INFO (coreZkRegister-2435-thread-1-processing-n:127.0.0.1:51812_ x:collection1 c:collection1) [n:127.0.0.1:51812_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:51812/collection1/ shard2
[junit4] 2> 846054 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_5FBDCDA8573442E0-001/shard-2-001/cores/collection1
[junit4] 2> 846054 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_5FBDCDA8573442E0-001/shard-2-001
[junit4] 2> 846055 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 846057 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@59302569{/,null,AVAILABLE}
[junit4] 2> 846057 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@34090090{HTTP/1.1,[http/1.1]}{127.0.0.1:55160}
[junit4] 2> 846058 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.e.j.s.Server Started @849187ms
[junit4] 2> 846058 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:50475/hdfs__localhost_50475__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-6.6_checkout_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsUnloadDistributedZkTest_5FBDCDA8573442E0-001_tempDir-002_jetty2, solrconfig=solrconfig.xml, hostContext=/, hostPort=55160, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_5FBDCDA8573442E0-001/shard-2-001/cores}
[junit4] 2> 846058 ERROR (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 846059 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.6.4
[junit4] 2> 846059 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 846059 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 846059 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-03-08T13:16:23.671Z
[junit4] 2> 846060 INFO (zkCallback-1295-thread-1-processing-n:127.0.0.1:51812_) [n:127.0.0.1:51812_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 846069 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 846069 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_5FBDCDA8573442E0-001/shard-2-001/solr.xml
[junit4] 2> 846076 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 846077 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35511/solr
[junit4] 2> 846088 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [n:127.0.0.1:55160_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 846090 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [n:127.0.0.1:55160_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 846092 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [n:127.0.0.1:55160_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55160_
[junit4] 2> 846094 INFO (zkCallback-1289-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 846096 INFO (zkCallback-1285-thread-1-processing-n:127.0.0.1:46394_) [n:127.0.0.1:46394_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 846097 INFO (zkCallback-1295-thread-1-processing-n:127.0.0.1:51812_) [n:127.0.0.1:51812_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 846101 INFO (zkCallback-1301-thread-1-processing-n:127.0.0.1:55160_) [n:127.0.0.1:55160_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 846109 INFO (coreZkRegister-2435-thread-1-processing-n:127.0.0.1:51812_ x:collection1 c:collection1) [n:127.0.0.1:51812_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 846149 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [n:127.0.0.1:55160_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_5FBDCDA8573442E0-001/shard-2-001/cores
[junit4] 2> 846149 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [n:127.0.0.1:55160_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 846151 INFO (coreLoadExecutor-2451-thread-1-processing-n:127.0.0.1:55160_) [n:127.0.0.1:55160_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 846152 INFO (OverseerStateUpdate-99648469736620036-127.0.0.1:46394_-n_0000000000) [n:127.0.0.1:46394_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 846256 INFO (zkCallback-1295-thread-1-processing-n:127.0.0.1:51812_) [n:127.0.0.1:51812_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 846256 INFO (zkCallback-1301-thread-1-processing-n:127.0.0.1:55160_) [n:127.0.0.1:55160_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 847170 INFO (coreLoadExecutor-2451-thread-1-processing-n:127.0.0.1:55160_) [n:127.0.0.1:55160_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.4
[junit4] 2> 847191 INFO (coreLoadExecutor-2451-thread-1-processing-n:127.0.0.1:55160_) [n:127.0.0.1:55160_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 847321 INFO (coreLoadExecutor-2451-thread-1-processing-n:127.0.0.1:55160_) [n:127.0.0.1:55160_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 847335 INFO (coreLoadExecutor-2451-thread-1-processing-n:127.0.0.1:55160_) [n:127.0.0.1:55160_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1, trusted=true
[junit4] 2> 847335 INFO (coreLoadExecutor-2451-thread-1-processing-n:127.0.0.1:55160_) [n:127.0.0.1:55160_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:50475/solr_hdfs_home
[junit4] 2> 847335 INFO (coreLoadExecutor-2451-thread-1-processing-n:127.0.0.1:55160_) [n:127.0.0.1:55160_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 847335 INFO (coreLoadExecutor-2451-thread-1-processing-n:127.0.0.1:55160_) [n:127.0.0.1:55160_ c:collection1 x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 847335 INFO (coreLoadExecutor-2451-thread-1-processing-n:127.0.0.1:55160_) [n:127.0.0.1:55160_ c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_5FBDCDA8573442E0-001/shard-2-001/cores/collection1], dataDir=[hdfs://localhost:50475/solr_hdfs_home/collection1/core_node2/data/]
[junit4] 2> 847336 INFO (coreLoadExecutor-2451-thread-1-processing-n:127.0.0.1:55160_) [n:127.0.0.1:55160_ c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@33d2f39d
[junit4] 2> 847336 INFO (coreLoadExecutor-2451-thread-1-processing-n:127.0.0.1:55160_) [n:127.0.0.1:55160_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:50475/solr_hdfs_home/collection1/core_node2/data/snapshot_metadata
[junit4] 2> 847345 INFO (coreLoadExecutor-2451-thread-1-processing-n:127.0.0.1:55160_) [n:127.0.0.1:55160_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 847345 INFO (coreLoadExecutor-2451-thread-1-processing-n:127.0.0.1:55160_) [n:127.0.0.1:55160_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 847345 INFO (coreLoadExecutor-2451-thread-1-processing-n:127.0.0.1:55160_) [n:127.0.0.1:55160_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 847356 INFO (coreLoadExecutor-2451-thread-1-processing-n:127.0.0.1:55160_) [n:127.0.0.1:55160_ c:collection1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 847357 INFO (coreLoadExecutor-2451-thread-1-processing-n:127.0.0.1:55160_) [n:127.0.0.1:55160_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:50475/solr_hdfs_home/collection1/core_node2/data
[junit4] 2> 847380 INFO (coreLoadExecutor-2451-thread-1-processing-n:127.0.0.1:55160_) [n:127.0.0.1:55160_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:50475/solr_hdfs_home/collection1/core_node2/data/index
[junit4] 2> 847398 INFO (coreLoadExecutor-2451-thread-1-processing-n:127.0.0.1:55160_) [n:127.0.0.1:55160_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 847398 INFO (coreLoadExecutor-2451-thread-1-processing-n:127.0.0.1:55160_) [n:127.0.0.1:55160_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 847398 INFO (coreLoadExecutor-2451-thread-1-processing-n:127.0.0.1:55160_) [n:127.0.0.1:55160_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 847419 INFO (coreLoadExecutor-2451-thread-1-processing-n:127.0.0.1:55160_) [n:127.0.0.1:55160_ c:collection1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 847419 INFO (coreLoadExecutor-2451-thread-1-processing-n:127.0.0.1:55160_) [n:127.0.0.1:55160_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=26, maxMergeAtOnceExplicit=45, maxMergedSegmentMB=29.3125, floorSegmentMB=0.25390625, forceMergeDeletesPctAllowed=0.9764929896714247, segmentsPerTier=50.0, maxCFSSegmentSizeMB=1.5185546875, noCFSRatio=0.30202736206383585
[junit4] 2> 847446 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:43167 is added to blk_1073741827_1003{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-b59e5c3f-1dff-4764-9bf2-322b33d05f46:NORMAL:127.0.0.1:36367|RBW], ReplicaUC[[DISK]DS-f41f3bb6-b5cd-4078-8203-8b333fac5fc3:NORMAL:127.0.0.1:43167|RBW]]} size 0
[junit4] 2> 847447 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:36367 is added to blk_1073741827_1003{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-f41f3bb6-b5cd-4078-8203-8b333fac5fc3:NORMAL:127.0.0.1:43167|RBW], ReplicaUC[[DISK]DS-41fb63d0-80d6-461e-8463-0a2d5b3f9fb0:NORMAL:127.0.0.1:36367|FINALIZED]]} size 0
[junit4] 2> 847458 WARN (coreLoadExecutor-2451-thread-1-processing-n:127.0.0.1:55160_) [n:127.0.0.1:55160_ c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 847506 INFO (coreLoadExecutor-2451-thread-1-processing-n:127.0.0.1:55160_) [n:127.0.0.1:55160_ c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 847506 INFO (coreLoadExecutor-2451-thread-1-processing-n:127.0.0.1:55160_) [n:127.0.0.1:55160_ c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 847506 INFO (coreLoadExecutor-2451-thread-1-processing-n:127.0.0.1:55160_) [n:127.0.0.1:55160_ c:collection1 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 847517 INFO (coreLoadExecutor-2451-thread-1-processing-n:127.0.0.1:55160_) [n:127.0.0.1:55160_ c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 847517 INFO (coreLoadExecutor-2451-thread-1-processing-n:127.0.0.1:55160_) [n:127.0.0.1:55160_ c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 847523 INFO (coreLoadExecutor-2451-thread-1-processing-n:127.0.0.1:55160_) [n:127.0.0.1:55160_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=7, maxMergeAtOnceExplicit=6, maxMergedSegmentMB=32.333984375, floorSegmentMB=1.826171875, forceMergeDeletesPctAllowed=21.043811099767726, segmentsPerTier=14.0, maxCFSSegmentSizeMB=0.912109375, noCFSRatio=1.0
[junit4] 2> 847532 INFO (coreLoadExecutor-2451-thread-1-processing-n:127.0.0.1:55160_) [n:127.0.0.1:55160_ c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4dad7f83[collection1] main]
[junit4] 2> 847534 INFO (coreLoadExecutor-2451-thread-1-processing-n:127.0.0.1:55160_) [n:127.0.0.1:55160_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 847535 INFO (coreLoadExecutor-2451-thread-1-processing-n:127.0.0.1:55160_) [n:127.0.0.1:55160_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 847535 INFO (coreLoadExecutor-2451-thread-1-processing-n:127.0.0.1:55160_) [n:127.0.0.1:55160_ c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 847538 INFO (searcherExecutor-2452-thread-1-processing-n:127.0.0.1:55160_ x:collection1 c:collection1) [n:127.0.0.1:55160_ c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@4dad7f83[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 847540 INFO (coreLoadExecutor-2451-thread-1-processing-n:127.0.0.1:55160_) [n:127.0.0.1:55160_ c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1594375521070743552
[junit4] 2> 847555 INFO (coreZkRegister-2446-thread-1-processing-n:127.0.0.1:55160_ x:collection1 c:collection1) [n:127.0.0.1:55160_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 847555 INFO (coreZkRegister-2446-thread-1-processing-n:127.0.0.1:55160_ x:collection1 c:collection1) [n:127.0.0.1:55160_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 847555 INFO (coreZkRegister-2446-thread-1-processing-n:127.0.0.1:55160_ x:collection1 c:collection1) [n:127.0.0.1:55160_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:55160/collection1/
[junit4] 2> 847556 INFO (coreZkRegister-2446-thread-1-processing-n:127.0.0.1:55160_ x:collection1 c:collection1) [n:127.0.0.1:55160_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 847556 INFO (coreZkRegister-2446-thread-1-processing-n:127.0.0.1:55160_ x:collection1 c:collection1) [n:127.0.0.1:55160_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:55160/collection1/ has no replicas
[junit4] 2> 847556 INFO (coreZkRegister-2446-thread-1-processing-n:127.0.0.1:55160_ x:collection1 c:collection1) [n:127.0.0.1:55160_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 847563 INFO (coreZkRegister-2446-thread-1-processing-n:127.0.0.1:55160_ x:collection1 c:collection1) [n:127.0.0.1:55160_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:55160/collection1/ shard1
[junit4] 2> 847660 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_5FBDCDA8573442E0-001/shard-3-001/cores/collection1
[junit4] 2> 847661 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_5FBDCDA8573442E0-001/shard-3-001
[junit4] 2> 847661 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 847663 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5827ebd3{/,null,AVAILABLE}
[junit4] 2> 847663 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@5b6011f{HTTP/1.1,[http/1.1]}{127.0.0.1:49401}
[junit4] 2> 847663 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.e.j.s.Server Started @850792ms
[junit4] 2> 847663 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:50475/hdfs__localhost_50475__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-6.6_checkout_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsUnloadDistributedZkTest_5FBDCDA8573442E0-001_tempDir-002_jetty3, solrconfig=solrconfig.xml, hostContext=/, hostPort=49401, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_5FBDCDA8573442E0-001/shard-3-001/cores}
[junit4] 2> 847663 ERROR (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 847671 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.6.4
[junit4] 2> 847671 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 847671 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 847671 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-03-08T13:16:25.283Z
[junit4] 2> 847672 INFO (zkCallback-1301-thread-1-processing-n:127.0.0.1:55160_) [n:127.0.0.1:55160_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 847672 INFO (zkCallback-1295-thread-1-processing-n:127.0.0.1:51812_) [n:127.0.0.1:51812_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 847682 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 847682 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_5FBDCDA8573442E0-001/shard-3-001/solr.xml
[junit4] 2> 847690 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 847691 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35511/solr
[junit4] 2> 847703 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [n:127.0.0.1:49401_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 847706 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [n:127.0.0.1:49401_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 847708 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [n:127.0.0.1:49401_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49401_
[junit4] 2> 847711 INFO (zkCallback-1301-thread-1-processing-n:127.0.0.1:55160_) [n:127.0.0.1:55160_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 847711 INFO (zkCallback-1289-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 847712 INFO (zkCallback-1285-thread-1-processing-n:127.0.0.1:46394_) [n:127.0.0.1:46394_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 847712 INFO (zkCallback-1295-thread-1-processing-n:127.0.0.1:51812_) [n:127.0.0.1:51812_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 847720 INFO (zkCallback-1307-thread-1-processing-n:127.0.0.1:49401_) [n:127.0.0.1:49401_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 847721 INFO (coreZkRegister-2446-thread-1-processing-n:127.0.0.1:55160_ x:collection1 c:collection1) [n:127.0.0.1:55160_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 847810 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [n:127.0.0.1:49401_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_5FBDCDA8573442E0-001/shard-3-001/cores
[junit4] 2> 847810 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[5FBDCDA8573442E0]) [n:127.0.0.1:49401_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 847812 INFO (coreLoadExecutor-2462-thread-1-processing-n:127.0.0.1:49401_) [n:127.0.0.1:49401_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 847820 INFO (OverseerStateUpdate-99648469736620036-127.0.0.1:46394_-n_0000000000) [n:127.0.0.1:46394_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 847923 INFO (zkCallback-1295-thread-1-processing-n:127.0.0.1:51812_) [n:127.0.0.1:51812_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 847923 INFO (zkCallback-1301-thread-1-processing-n:127.0.0.1:55160_) [n:127.0.0.1:55160_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 847923 INFO (zkCallback-1307-thread-1-processing-n:127.0.0.1:49401_) [n:127.0.0.1:49401_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 848840 INFO (coreLoadExecutor-2462-thread-1-processing-n:127.0.0.1:49401_) [n:127.0.0.1:49401_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.4
[junit4] 2> 848868 INFO (coreLoadExecutor-2462-thread-1-processing-n:127.0.0.1:49401_) [n:127.0.0.1:49401_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 848997 INFO (coreLoadExecutor-2462-thread-1-processing-n:127.0.0.1:49401_) [n:127.0.0.1:49401_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 849011 INFO (coreLoadExecutor-2462-thread-1-processing-n:127.0.0.1:49401_) [n:127.0.0.1:49401_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1, trusted=true
[junit4] 2> 849012 INFO (coreLoadExecutor-2462-thread-1-processing-n:127.0.0.1:49401_) [n:127.0.0.1:49401_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:50475/solr_hdfs_home
[junit4] 2> 849012 INFO (coreLoadExecutor-2462-thread-1-processing-n:127.0.0.1:49401_) [n:127.0.0.1:49401_ c:collectio
[...truncated too long message...]
toryScanner: shutdown has been called
[junit4] 2> 3999434 INFO (SUITE-StressHdfsTest-seed#[5FBDCDA8573442E0]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
[junit4] 2> 3999535 WARN (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_5FBDCDA8573442E0-001/tempDir-001/hdfsBaseDir/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_5FBDCDA8573442E0-001/tempDir-001/hdfsBaseDir/data/data4/]] heartbeating to localhost/127.0.0.1:44368) [ ] o.a.h.h.s.d.IncrementalBlockReportManager IncrementalBlockReportManager interrupted
[junit4] 2> 3999536 WARN (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_5FBDCDA8573442E0-001/tempDir-001/hdfsBaseDir/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_5FBDCDA8573442E0-001/tempDir-001/hdfsBaseDir/data/data4/]] heartbeating to localhost/127.0.0.1:44368) [ ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-1812823391-10.41.0.5-1520517966047 (Datanode Uuid c856dec8-3b55-4fa1-b966-dd8c3f89f862) service to localhost/127.0.0.1:44368
[junit4] 2> 3999538 WARN (SUITE-StressHdfsTest-seed#[5FBDCDA8573442E0]-worker) [ ] o.a.h.h.s.d.DirectoryScanner DirectoryScanner: shutdown has been called
[junit4] 2> 3999561 INFO (SUITE-StressHdfsTest-seed#[5FBDCDA8573442E0]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
[junit4] 2> 3999663 WARN (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_5FBDCDA8573442E0-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_5FBDCDA8573442E0-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to localhost/127.0.0.1:44368) [ ] o.a.h.h.s.d.IncrementalBlockReportManager IncrementalBlockReportManager interrupted
[junit4] 2> 3999663 WARN (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_5FBDCDA8573442E0-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_5FBDCDA8573442E0-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to localhost/127.0.0.1:44368) [ ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-1812823391-10.41.0.5-1520517966047 (Datanode Uuid 1ea4f383-56b4-4097-9cc8-773136115633) service to localhost/127.0.0.1:44368
[junit4] 2> 3999674 INFO (SUITE-StressHdfsTest-seed#[5FBDCDA8573442E0]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
[junit4] 2> 3999777 INFO (SUITE-StressHdfsTest-seed#[5FBDCDA8573442E0]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_5FBDCDA8573442E0-001
[junit4] 2> Mar 08, 2018 2:08:57 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 34 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): {range_facet_l_dv=BlockTreeOrds(blocksize=128), multiDefault=FST50, intDefault=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), id=BlockTreeOrds(blocksize=128), range_facet_i_dv=FST50, text=Lucene50(blocksize=128), range_facet_l=FST50, timestamp=FST50, txt_t=Lucene50(blocksize=128)}, docValues:{range_facet_l_dv=DocValuesFormat(name=Memory), _version_=DocValuesFormat(name=Lucene54), range_facet_i_dv=DocValuesFormat(name=Lucene54), intDvoDefault=DocValuesFormat(name=Memory), timestamp=DocValuesFormat(name=Lucene54)}, maxPointsInLeafNode=1009, maxMBSortInHeap=5.078668139022737, sim=RandomSimilarity(queryNorm=false,coord=yes): {}, locale=pl, timezone=US/Arizona
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=2,free=206427680,total=535298048
[junit4] 2> NOTE: All tests run in this JVM: [TestSchemaVersionResource, BitVectorTest, RequiredFieldsTest, ScriptEngineTest, CurrencyFieldOpenExchangeTest, TestQuerySenderNoQuery, TestDistribDocBasedVersion, TestUninvertingReader, QueryResultKeyTest, ResourceLoaderTest, NumericFieldsTest, UUIDFieldTest, LeaderElectionContextKeyTest, TestNamedUpdateProcessors, TestHighlightDedupGrouping, TestSchemaNameResource, UtilsToolTest, TestSolrCloudWithKerberosAlt, V2ApiIntegrationTest, DirectoryFactoryTest, TestFunctionQuery, TestCorePropertiesReload, TestValueSourceCache, TestManagedSynonymFilterFactory, TestStressRecovery, TestJettySolrRunner, TestSolrDeletionPolicy2, BlockDirectoryTest, TestSubQueryTransformer, TestSortByMinMaxFunction, CollectionReloadTest, SolrGangliaReporterTest, CollectionsAPIAsyncDistributedZkTest, TestPathTrie, ZkNodePropsTest, TestSolrQueryParserResource, SpatialRPTFieldTypeTest, TestHdfsBackupRestoreCore, TestConfigsApi, DocValuesNotIndexedTest, CdcrReplicationHandlerTest, TestCrossCoreJoin, TestExpandComponent, SimpleMLTQParserTest, TestFreeTextSuggestions, OverseerRolesTest, CdcrUpdateLogTest, DistributedFacetPivotLongTailTest, DistributedExpandComponentTest, SolrRequestParserTest, TestIBSimilarityFactory, CustomCollectionTest, HLLSerializationTest, TestCloudNestedDocsSort, SolrIndexMetricsTest, TestCloudPseudoReturnFields, TestLegacyFieldCache, TestDocSet, TestLazyCores, TestSolr4Spatial, DateFieldTest, PeerSyncReplicationTest, TestAnalyzeInfixSuggestions, PeerSyncTest, BaseCdcrDistributedZkTest, SynonymTokenizerTest, TestStressInPlaceUpdates, BlockJoinFacetRandomTest, DefaultValueUpdateProcessorTest, ChangedSchemaMergeTest, RegexBoostProcessorTest, TestFastLRUCache, BasicAuthIntegrationTest, TestConfigSetImmutable, ExternalFileFieldSortTest, TestSimpleTextCodec, TestReplicationHandler, TestDistributedSearch, TestFieldCacheSortRandom, TestOmitPositions, TestJsonFacets, TestBulkSchemaAPI, ShardSplitTest, DistributedFacetPivotWhiteBoxTest, TestXmlQParser, TestCollectionAPI, BasicZkTest, TestSolr4Spatial2, HdfsTlogReplayBufferedWhileIndexingTest, TestRandomDVFaceting, TestConfigSetsAPIZkFailure, VMParamsZkACLAndCredentialsProvidersTest, TestSha256AuthenticationProvider, TestMacroExpander, UpdateLogTest, HdfsNNFailoverTest, TestSolrXml, MultiThreadedOCPTest, TestQueryWrapperFilter, ConfigSetsAPITest, DeleteNodeTest, TestFieldResource, FileBasedSpellCheckerTest, SubstringBytesRefFilterTest, TestAtomicUpdateErrorCases, TestSchemalessBufferedUpdates, ClusterStateTest, TestRemoteStreaming, TestSolrCoreProperties, MetricUtilsTest, TestDocumentBuilder, PreAnalyzedUpdateProcessorTest, TestBadConfig, LeaderElectionIntegrationTest, RecoveryZkTest, TestRandomFaceting, TestZkChroot, AutoCommitTest, TestFaceting, TestRecovery, TestHashPartitioner, DistributedSpellCheckComponentTest, TermVectorComponentDistributedTest, TestRealTimeGet, TestReload, TestCoreContainer, SolrCoreTest, SolrCmdDistributorTest, QueryElevationComponentTest, BadIndexSchemaTest, TestFiltering, ShowFileRequestHandlerTest, DistributedQueryElevationComponentTest, SolrIndexSplitterTest, AnalysisAfterCoreReloadTest, SignatureUpdateProcessorFactoryTest, SpellCheckCollatorTest, TestFoldingMultitermQuery, DocValuesTest, TestStressLucene, TestTrie, TestCSVLoader, NoCacheHeaderTest, WordBreakSolrSpellCheckerTest, TestPseudoReturnFields, QueryEqualityTest, SortByFunctionTest, XsltUpdateRequestHandlerTest, CacheHeaderTest, StandardRequestHandlerTest, DocumentAnalysisRequestHandlerTest, XmlUpdateRequestHandlerTest, TermVectorComponentTest, TestSolrQueryParser, LoggingHandlerTest, TestJmxIntegration, MBeansHandlerTest, TestAnalyzedSuggestions, JsonLoaderTest, TestPartialUpdateDeduplication, PingRequestHandlerTest, TestBinaryResponseWriter, HighlighterConfigTest, UpdateParamsTest, AlternateDirectoryTest, TestSolrIndexConfig, CopyFieldTest, BadComponentTest, TestBinaryField, TestElisionMultitermQuery, TestPostingsSolrHighlighter, DirectSolrConnectionTest, TestPhraseSuggestions, SpellPossibilityIteratorTest, TestCodecSupport, TestLMJelinekMercerSimilarityFactory, TestFastWriter, TestJmxMonitoredMap, TestSystemIdResolver, PrimUtilsTest, TestSuggestSpellingConverter, RAMDirectoryFactoryTest, TestSolrJ, TestLRUCache, SliceStateTest, SystemInfoHandlerTest, CircularListTest, TestRTGBase, CursorPagingTest, SolrTestCaseJ4Test, TestCursorMarkWithoutUniqueKey, TestDistributedMissingSort, TestSimpleTrackingShardHandler, TestTolerantSearch, AliasIntegrationTest, AsyncCallRequestStatusResponseTest, CdcrRequestHandlerTest, DeleteLastCustomShardedReplicaTest, DeleteShardTest, DistribCursorPagingTest, LeaderFailoverAfterPartitionTest, LeaderInitiatedRecoveryOnShardRestartTest, MigrateRouteKeyTest, NodeMutatorTest, OutOfBoxZkACLAndCredentialsProvidersTest, OverseerCollectionConfigSetProcessorTest, RemoteQueryErrorTest, ReplicationFactorTest, RollingRestartTest, TestRebalanceLeaders, TestSegmentSorting, TestSolrCloudWithDelegationTokens, TestTolerantUpdateProcessorRandomCloud, TriLevelCompositeIdRoutingTest, HdfsThreadLeakTest, HdfsWriteToMultipleCollectionsTest, StressHdfsTest]
[junit4] Completed [564/714 (3!)] on J0 in 181.20s, 1 test, 1 error <<< FAILURES!
[...truncated 46772 lines...]
[JENKINS] Lucene-Solr-NightlyTests-6.6 - Build # 37 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.6/37/
5 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsBasicDistributedZkTest
Error Message:
ObjectTracker found 1 object(s) that were not released!!! [NRTCachingDirectory] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347) at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:353) at org.apache.solr.core.SolrCore.cleanupOldIndexDirectories(SolrCore.java:3044) at org.apache.solr.core.SolrCore.close(SolrCore.java:1575) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:969) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:830) at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:950) at org.apache.solr.core.CoreContainer.lambda$load$5(CoreContainer.java:565) at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
java.lang.AssertionError: ObjectTracker found 1 object(s) that were not released!!! [NRTCachingDirectory]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:353)
at org.apache.solr.core.SolrCore.cleanupOldIndexDirectories(SolrCore.java:3044)
at org.apache.solr.core.SolrCore.close(SolrCore.java:1575)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:969)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:830)
at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:950)
at org.apache.solr.core.CoreContainer.lambda$load$5(CoreContainer.java:565)
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([DF5C6277AD21161E]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:302)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.hdfs.HdfsSyncSliceTest.test
Error Message:
Error from server at https://127.0.0.1:40361/_dp/e/collection1: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 500 </title> </head> <body> <h2>HTTP ERROR: 500</h2> <p>Problem accessing /_dp/e/collection1/update. Reason: <pre> {metadata={error-class=org.apache.solr.core.SolrCoreInitializationException,root-error-class=java.lang.OutOfMemoryError},msg=SolrCore 'collection1' is not available due to init failure: The max direct memory is likely too low. Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~8.388608 MB.,trace=org.apache.solr.core.SolrCoreInitializationException: SolrCore 'collection1' is not available due to init failure: The max direct memory is likely too low. Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~8.388608 MB. at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1361) at org.apache.solr.servlet.HttpSolrCall.init(HttpSolrCall.java:268) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:483) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:361) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:305) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:136) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.eclipse.jetty.server.Server.handle(Server.java:534) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:202) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.solr.common.SolrException: The max direct memory is likely too low. Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~8.388608 MB. at org.apache.solr.core.SolrCore.<init>(SolrCore.java:977) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:830) at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:950) at org.apache.solr.core.CoreContainer.lambda$load$5(CoreContainer.java:565) at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ... 1 more Caused by: java.lang.RuntimeException: The max direct memory is likely too low. Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~8.388608 MB. at org.apache.solr.core.HdfsDirectoryFactory.createBlockCache(HdfsDirectoryFactory.java:312) at org.apache.solr.core.HdfsDirectoryFactory.getBlockDirectoryCache(HdfsDirectoryFactory.java:288) at org.apache.solr.core.HdfsDirectoryFactory.create(HdfsDirectoryFactory.java:228) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:346) at org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:482) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:917) ... 9 more Caused by: java.lang.OutOfMemoryError: Direct buffer memory at java.nio.Bits.reserveMemory(Bits.java:694) at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123) at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311) at org.apache.solr.store.blockcache.BlockCache.<init>(BlockCache.java:75) at org.apache.solr.core.HdfsDirectoryFactory.createBlockCache(HdfsDirectoryFactory.java:310) ... 14 more ,code=500}</pre></p> <hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.14.v20161028</a><hr/> </body> </html>
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:40361/_dp/e/collection1: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /_dp/e/collection1/update. Reason:
<pre> {metadata={error-class=org.apache.solr.core.SolrCoreInitializationException,root-error-class=java.lang.OutOfMemoryError},msg=SolrCore 'collection1' is not available due to init failure: The max direct memory is likely too low. Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~8.388608 MB.,trace=org.apache.solr.core.SolrCoreInitializationException: SolrCore 'collection1' is not available due to init failure: The max direct memory is likely too low. Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~8.388608 MB.
at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:1361)
at org.apache.solr.servlet.HttpSolrCall.init(HttpSolrCall.java:268)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:483)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:361)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:305)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:136)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:534)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:202)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.common.SolrException: The max direct memory is likely too low. Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~8.388608 MB.
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:977)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:830)
at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:950)
at org.apache.solr.core.CoreContainer.lambda$load$5(CoreContainer.java:565)
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
Caused by: java.lang.RuntimeException: The max direct memory is likely too low. Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~8.388608 MB.
at org.apache.solr.core.HdfsDirectoryFactory.createBlockCache(HdfsDirectoryFactory.java:312)
at org.apache.solr.core.HdfsDirectoryFactory.getBlockDirectoryCache(HdfsDirectoryFactory.java:288)
at org.apache.solr.core.HdfsDirectoryFactory.create(HdfsDirectoryFactory.java:228)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:346)
at org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:482)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:917)
... 9 more
Caused by: java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Bits.java:694)
at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
at org.apache.solr.store.blockcache.BlockCache.<init>(BlockCache.java:75)
at org.apache.solr.core.HdfsDirectoryFactory.createBlockCache(HdfsDirectoryFactory.java:310)
... 14 more
,code=500}</pre></p>
<hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.14.v20161028</a><hr/>
</body>
</html>
at __randomizedtesting.SeedInfo.seed([DF5C6277AD21161E:57085DAD03DD7BE6]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:580)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
at org.apache.solr.client.solrj.SolrClient.deleteByQuery(SolrClient.java:895)
at org.apache.solr.client.solrj.SolrClient.deleteByQuery(SolrClient.java:858)
at org.apache.solr.client.solrj.SolrClient.deleteByQuery(SolrClient.java:873)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.del(AbstractFullDistribZkTestBase.java:835)
at org.apache.solr.cloud.SyncSliceTest.test(SyncSliceTest.java:73)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.core.TestDynamicLoading.testDynamicLoading
Error Message:
Could not get expected value 'org.apache.solr.core.BlobStoreTestRequestHandler' for path 'overlay/requestHandler/\/test1/class' full output: { "responseHeader":{ "status":0, "QTime":0}, "overlay":{ "znodeVersion":0, "runtimeLib":{"colltest":{ "name":"colltest", "version":1}}}}, from server: null
Stack Trace:
java.lang.AssertionError: Could not get expected value 'org.apache.solr.core.BlobStoreTestRequestHandler' for path 'overlay/requestHandler/\/test1/class' full output: {
"responseHeader":{
"status":0,
"QTime":0},
"overlay":{
"znodeVersion":0,
"runtimeLib":{"colltest":{
"name":"colltest",
"version":1}}}}, from server: null
at __randomizedtesting.SeedInfo.seed([DF5C6277AD21161E:7114F205AFCB3BE]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.core.TestSolrConfigHandler.testForResponseElement(TestSolrConfigHandler.java:556)
at org.apache.solr.core.TestDynamicLoading.testDynamicLoading(TestDynamicLoading.java:97)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload
Error Message:
expected:<[{indexVersion=1520441148100,generation=2,filelist=[_3my.cfe, _3my.cfs, _3my.si, _3nx.cfe, _3nx.cfs, _3nx.si, _3ny.fdt, _3ny.fdx, _3ny.fnm, _3ny.nvd, _3ny.nvm, _3ny.si, _3ny_Memory_0.ram, _3nz.fdt, _3nz.fdx, _3nz.fnm, _3nz.nvd, _3nz.nvm, _3nz.si, _3nz_Memory_0.ram, _3o1.fdt, _3o1.fdx, _3o1.fnm, _3o1.nvd, _3o1.nvm, _3o1.si, _3o1_Memory_0.ram, segments_2]}]> but was:<[{indexVersion=1520441148100,generation=2,filelist=[_3my.cfe, _3my.cfs, _3my.si, _3nx.cfe, _3nx.cfs, _3nx.si, _3ny.fdt, _3ny.fdx, _3ny.fnm, _3ny.nvd, _3ny.nvm, _3ny.si, _3ny_Memory_0.ram, _3nz.fdt, _3nz.fdx, _3nz.fnm, _3nz.nvd, _3nz.nvm, _3nz.si, _3nz_Memory_0.ram, _3o1.fdt, _3o1.fdx, _3o1.fnm, _3o1.nvd, _3o1.nvm, _3o1.si, _3o1_Memory_0.ram, segments_2]}, {indexVersion=1520441148100,generation=3,filelist=[_3o0.cfe, _3o0.cfs, _3o0.si, _3o1.fdt, _3o1.fdx, _3o1.fnm, _3o1.nvd, _3o1.nvm, _3o1.si, _3o1_Memory_0.ram, _3o2.fdt, _3o2.fdx, _3o2.fnm, _3o2.nvd, _3o2.nvm, _3o2.si, _3o2_Memory_0.ram, segments_3]}]>
Stack Trace:
java.lang.AssertionError: expected:<[{indexVersion=1520441148100,generation=2,filelist=[_3my.cfe, _3my.cfs, _3my.si, _3nx.cfe, _3nx.cfs, _3nx.si, _3ny.fdt, _3ny.fdx, _3ny.fnm, _3ny.nvd, _3ny.nvm, _3ny.si, _3ny_Memory_0.ram, _3nz.fdt, _3nz.fdx, _3nz.fnm, _3nz.nvd, _3nz.nvm, _3nz.si, _3nz_Memory_0.ram, _3o1.fdt, _3o1.fdx, _3o1.fnm, _3o1.nvd, _3o1.nvm, _3o1.si, _3o1_Memory_0.ram, segments_2]}]> but was:<[{indexVersion=1520441148100,generation=2,filelist=[_3my.cfe, _3my.cfs, _3my.si, _3nx.cfe, _3nx.cfs, _3nx.si, _3ny.fdt, _3ny.fdx, _3ny.fnm, _3ny.nvd, _3ny.nvm, _3ny.si, _3ny_Memory_0.ram, _3nz.fdt, _3nz.fdx, _3nz.fnm, _3nz.nvd, _3nz.nvm, _3nz.si, _3nz_Memory_0.ram, _3o1.fdt, _3o1.fdx, _3o1.fnm, _3o1.nvd, _3o1.nvm, _3o1.si, _3o1_Memory_0.ram, segments_2]}, {indexVersion=1520441148100,generation=3,filelist=[_3o0.cfe, _3o0.cfs, _3o0.si, _3o1.fdt, _3o1.fdx, _3o1.fnm, _3o1.nvd, _3o1.nvm, _3o1.si, _3o1_Memory_0.ram, _3o2.fdt, _3o2.fdx, _3o2.fnm, _3o2.nvd, _3o2.nvm, _3o2.si, _3o2_Memory_0.ram, segments_3]}]>
at __randomizedtesting.SeedInfo.seed([DF5C6277AD21161E:FA8B7947DD69181D]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:147)
at org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload(TestReplicationHandler.java:1281)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.hdfs.HdfsUnloadDistributedZkTest.test
Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:40687/r/ng
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:40687/r/ng
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:623)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at org.apache.solr.cloud.UnloadDistributedZkTest.testUnloadShardAndCollection(UnloadDistributedZkTest.java:134)
at org.apache.solr.cloud.UnloadDistributedZkTest.test(UnloadDistributedZkTest.java:71)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:515)
... 46 more
Build Log:
[...truncated 12306 lines...]
[junit4] Suite: org.apache.solr.cloud.hdfs.HdfsUnloadDistributedZkTest
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_DF5C6277AD21161E-001/init-core-data-001
[junit4] 2> 658009 WARN (SUITE-HdfsUnloadDistributedZkTest-seed#[DF5C6277AD21161E]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=8 numCloses=8
[junit4] 2> 658010 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[DF5C6277AD21161E]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields
[junit4] 2> 658011 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[DF5C6277AD21161E]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
[junit4] 2> 658011 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[DF5C6277AD21161E]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /r/ng
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 658550 WARN (SUITE-HdfsUnloadDistributedZkTest-seed#[DF5C6277AD21161E]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 658630 WARN (SUITE-HdfsUnloadDistributedZkTest-seed#[DF5C6277AD21161E]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 658653 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[DF5C6277AD21161E]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 658764 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[DF5C6277AD21161E]-worker) [ ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/hdfs to ./temp/Jetty_lucene2.us.west_apache_org_34818_hdfs____.p27vaw/webapp
[junit4] 2> 660656 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[DF5C6277AD21161E]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@lucene2-us-west.apache.org:34818
[junit4] 2> 661948 WARN (SUITE-HdfsUnloadDistributedZkTest-seed#[DF5C6277AD21161E]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 661949 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[DF5C6277AD21161E]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 662181 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[DF5C6277AD21161E]-worker) [ ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_43475_datanode____4qzb8a/webapp
[junit4] 2> 663520 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[DF5C6277AD21161E]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43475
[junit4] 2> 664594 WARN (SUITE-HdfsUnloadDistributedZkTest-seed#[DF5C6277AD21161E]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 664611 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[DF5C6277AD21161E]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 664793 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[DF5C6277AD21161E]-worker) [ ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_43791_datanode____b7aaaf/webapp
[junit4] 2> 669347 ERROR (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_DF5C6277AD21161E-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_DF5C6277AD21161E-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:41587) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 669607 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x9d9e77a5f62b8: from storage DS-b1adebb0-2962-402e-bba2-3aa07a7764fc node DatanodeRegistration(127.0.0.1:34423, datanodeUuid=8f2f064c-01f8-4041-a2ec-e3b933313ec7, infoPort=45142, infoSecurePort=0, ipcPort=38295, storageInfo=lv=-56;cid=testClusterID;nsid=1649085565;c=0), blocks: 0, hasStaleStorage: true, processing time: 126 msecs
[junit4] 2> 669608 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x9d9e77a5f62b8: from storage DS-53a096d0-87ee-4d67-8439-dae5169d8a99 node DatanodeRegistration(127.0.0.1:34423, datanodeUuid=8f2f064c-01f8-4041-a2ec-e3b933313ec7, infoPort=45142, infoSecurePort=0, ipcPort=38295, storageInfo=lv=-56;cid=testClusterID;nsid=1649085565;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 670824 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[DF5C6277AD21161E]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43791
[junit4] 2> 672349 ERROR (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_DF5C6277AD21161E-001/tempDir-001/hdfsBaseDir/data/data3/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_DF5C6277AD21161E-001/tempDir-001/hdfsBaseDir/data/data4/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:41587) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 672418 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x9d9e82a0e2219: from storage DS-c9a73587-50f4-419c-8404-b3969f878909 node DatanodeRegistration(127.0.0.1:43167, datanodeUuid=33ad7626-df79-47a6-ba5f-d00f30ccf677, infoPort=34284, infoSecurePort=0, ipcPort=40891, storageInfo=lv=-56;cid=testClusterID;nsid=1649085565;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 672419 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x9d9e82a0e2219: from storage DS-57f9dfe6-6066-4bf2-9121-7804eedc1ebd node DatanodeRegistration(127.0.0.1:43167, datanodeUuid=33ad7626-df79-47a6-ba5f-d00f30ccf677, infoPort=34284, infoSecurePort=0, ipcPort=40891, storageInfo=lv=-56;cid=testClusterID;nsid=1649085565;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 673176 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 673265 INFO (Thread-8879) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 673265 INFO (Thread-8879) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 673290 ERROR (Thread-8879) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 673366 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.a.s.c.ZkTestServer start zk server on port:37866
[junit4] 2> 673524 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 673538 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 673539 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 673541 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 673554 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 673555 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 673556 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 673557 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 673574 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 673575 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 673591 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 674441 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_DF5C6277AD21161E-001/control-001/cores/collection1
[junit4] 2> 674455 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 674503 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5e6b2437{/r/ng,null,AVAILABLE}
[junit4] 2> 674503 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@281226c2{HTTP/1.1,[http/1.1]}{127.0.0.1:45189}
[junit4] 2> 674503 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.e.j.s.Server Started @686472ms
[junit4] 2> 674503 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://lucene2-us-west.apache.org:41587/hdfs__lucene2-us-west.apache.org_41587__home_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-6.6_checkout_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsUnloadDistributedZkTest_DF5C6277AD21161E-001_tempDir-002_control_data, hostContext=/r/ng, hostPort=45189, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_DF5C6277AD21161E-001/control-001/cores}
[junit4] 2> 674503 ERROR (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 674503 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.6.4
[junit4] 2> 674503 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 674504 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 674504 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-03-07T15:11:07.101Z
[junit4] 2> 674631 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 674631 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_DF5C6277AD21161E-001/control-001/solr.xml
[junit4] 2> 674692 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 674695 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37866/solr
[junit4] 2> 674963 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [n:127.0.0.1:45189_r%2Fng ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 674980 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [n:127.0.0.1:45189_r%2Fng ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:45189_r%2Fng
[junit4] 2> 674981 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [n:127.0.0.1:45189_r%2Fng ] o.a.s.c.Overseer Overseer (id=99643258690797572-127.0.0.1:45189_r%2Fng-n_0000000000) starting
[junit4] 2> 675036 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [n:127.0.0.1:45189_r%2Fng ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45189_r%2Fng
[junit4] 2> 675050 INFO (zkCallback-158-thread-1-processing-n:127.0.0.1:45189_r%2Fng) [n:127.0.0.1:45189_r%2Fng ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 675267 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [n:127.0.0.1:45189_r%2Fng ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_DF5C6277AD21161E-001/control-001/cores
[junit4] 2> 675267 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [n:127.0.0.1:45189_r%2Fng ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 675305 INFO (coreLoadExecutor-306-thread-1-processing-n:127.0.0.1:45189_r%2Fng) [n:127.0.0.1:45189_r%2Fng ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 675343 INFO (OverseerStateUpdate-99643258690797572-127.0.0.1:45189_r%2Fng-n_0000000000) [n:127.0.0.1:45189_r%2Fng ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 676571 INFO (coreLoadExecutor-306-thread-1-processing-n:127.0.0.1:45189_r%2Fng) [n:127.0.0.1:45189_r%2Fng c:control_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.4
[junit4] 2> 676636 INFO (coreLoadExecutor-306-thread-1-processing-n:127.0.0.1:45189_r%2Fng) [n:127.0.0.1:45189_r%2Fng c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 677409 INFO (coreLoadExecutor-306-thread-1-processing-n:127.0.0.1:45189_r%2Fng) [n:127.0.0.1:45189_r%2Fng c:control_collection x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 677710 INFO (coreLoadExecutor-306-thread-1-processing-n:127.0.0.1:45189_r%2Fng) [n:127.0.0.1:45189_r%2Fng c:control_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection, trusted=true
[junit4] 2> 677711 INFO (coreLoadExecutor-306-thread-1-processing-n:127.0.0.1:45189_r%2Fng) [n:127.0.0.1:45189_r%2Fng c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://lucene2-us-west.apache.org:41587/solr_hdfs_home
[junit4] 2> 677711 INFO (coreLoadExecutor-306-thread-1-processing-n:127.0.0.1:45189_r%2Fng) [n:127.0.0.1:45189_r%2Fng c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 677711 INFO (coreLoadExecutor-306-thread-1-processing-n:127.0.0.1:45189_r%2Fng) [n:127.0.0.1:45189_r%2Fng c:control_collection x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 677711 INFO (coreLoadExecutor-306-thread-1-processing-n:127.0.0.1:45189_r%2Fng) [n:127.0.0.1:45189_r%2Fng c:control_collection x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_DF5C6277AD21161E-001/control-001/cores/collection1], dataDir=[hdfs://lucene2-us-west.apache.org:41587/solr_hdfs_home/control_collection/core_node1/data/]
[junit4] 2> 677712 INFO (coreLoadExecutor-306-thread-1-processing-n:127.0.0.1:45189_r%2Fng) [n:127.0.0.1:45189_r%2Fng c:control_collection x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1397dcb1
[junit4] 2> 677728 INFO (coreLoadExecutor-306-thread-1-processing-n:127.0.0.1:45189_r%2Fng) [n:127.0.0.1:45189_r%2Fng c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:41587/solr_hdfs_home/control_collection/core_node1/data/snapshot_metadata
[junit4] 2> 677796 INFO (coreLoadExecutor-306-thread-1-processing-n:127.0.0.1:45189_r%2Fng) [n:127.0.0.1:45189_r%2Fng c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 677796 INFO (coreLoadExecutor-306-thread-1-processing-n:127.0.0.1:45189_r%2Fng) [n:127.0.0.1:45189_r%2Fng c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 677796 INFO (coreLoadExecutor-306-thread-1-processing-n:127.0.0.1:45189_r%2Fng) [n:127.0.0.1:45189_r%2Fng c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 679063 INFO (coreLoadExecutor-306-thread-1-processing-n:127.0.0.1:45189_r%2Fng) [n:127.0.0.1:45189_r%2Fng c:control_collection x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 679090 INFO (coreLoadExecutor-306-thread-1-processing-n:127.0.0.1:45189_r%2Fng) [n:127.0.0.1:45189_r%2Fng c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:41587/solr_hdfs_home/control_collection/core_node1/data
[junit4] 2> 679235 INFO (coreLoadExecutor-306-thread-1-processing-n:127.0.0.1:45189_r%2Fng) [n:127.0.0.1:45189_r%2Fng c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:41587/solr_hdfs_home/control_collection/core_node1/data/index
[junit4] 2> 679310 INFO (coreLoadExecutor-306-thread-1-processing-n:127.0.0.1:45189_r%2Fng) [n:127.0.0.1:45189_r%2Fng c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 679311 INFO (coreLoadExecutor-306-thread-1-processing-n:127.0.0.1:45189_r%2Fng) [n:127.0.0.1:45189_r%2Fng c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 679311 INFO (coreLoadExecutor-306-thread-1-processing-n:127.0.0.1:45189_r%2Fng) [n:127.0.0.1:45189_r%2Fng c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 680627 INFO (coreLoadExecutor-306-thread-1-processing-n:127.0.0.1:45189_r%2Fng) [n:127.0.0.1:45189_r%2Fng c:control_collection x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 680628 INFO (coreLoadExecutor-306-thread-1-processing-n:127.0.0.1:45189_r%2Fng) [n:127.0.0.1:45189_r%2Fng c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=43, maxMergeAtOnceExplicit=29, maxMergedSegmentMB=63.90234375, floorSegmentMB=0.830078125, forceMergeDeletesPctAllowed=3.5752316798021857, segmentsPerTier=25.0, maxCFSSegmentSizeMB=1.09375, noCFSRatio=0.7147959364477267
[junit4] 2> 680884 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:43167 is added to blk_1073741825_1001{UCState=COMMITTED, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-53a096d0-87ee-4d67-8439-dae5169d8a99:NORMAL:127.0.0.1:34423|RBW], ReplicaUC[[DISK]DS-57f9dfe6-6066-4bf2-9121-7804eedc1ebd:NORMAL:127.0.0.1:43167|RBW]]} size 71
[junit4] 2> 680884 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:34423 is added to blk_1073741825_1001 size 71
[junit4] 2> 681455 WARN (coreLoadExecutor-306-thread-1-processing-n:127.0.0.1:45189_r%2Fng) [n:127.0.0.1:45189_r%2Fng c:control_collection x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 682053 INFO (coreLoadExecutor-306-thread-1-processing-n:127.0.0.1:45189_r%2Fng) [n:127.0.0.1:45189_r%2Fng c:control_collection x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 682053 INFO (coreLoadExecutor-306-thread-1-processing-n:127.0.0.1:45189_r%2Fng) [n:127.0.0.1:45189_r%2Fng c:control_collection x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 682053 INFO (coreLoadExecutor-306-thread-1-processing-n:127.0.0.1:45189_r%2Fng) [n:127.0.0.1:45189_r%2Fng c:control_collection x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 682143 INFO (coreLoadExecutor-306-thread-1-processing-n:127.0.0.1:45189_r%2Fng) [n:127.0.0.1:45189_r%2Fng c:control_collection x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 682143 INFO (coreLoadExecutor-306-thread-1-processing-n:127.0.0.1:45189_r%2Fng) [n:127.0.0.1:45189_r%2Fng c:control_collection x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 682179 INFO (coreLoadExecutor-306-thread-1-processing-n:127.0.0.1:45189_r%2Fng) [n:127.0.0.1:45189_r%2Fng c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@77372943
[junit4] 2> 682451 INFO (coreLoadExecutor-306-thread-1-processing-n:127.0.0.1:45189_r%2Fng) [n:127.0.0.1:45189_r%2Fng c:control_collection x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1affd7e9[collection1] main]
[junit4] 2> 682511 INFO (coreLoadExecutor-306-thread-1-processing-n:127.0.0.1:45189_r%2Fng) [n:127.0.0.1:45189_r%2Fng c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 682547 INFO (coreLoadExecutor-306-thread-1-processing-n:127.0.0.1:45189_r%2Fng) [n:127.0.0.1:45189_r%2Fng c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 682547 INFO (coreLoadExecutor-306-thread-1-processing-n:127.0.0.1:45189_r%2Fng) [n:127.0.0.1:45189_r%2Fng c:control_collection x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 682630 INFO (searcherExecutor-307-thread-1-processing-n:127.0.0.1:45189_r%2Fng x:collection1 c:control_collection) [n:127.0.0.1:45189_r%2Fng c:control_collection x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1affd7e9[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 682656 INFO (coreLoadExecutor-306-thread-1-processing-n:127.0.0.1:45189_r%2Fng) [n:127.0.0.1:45189_r%2Fng c:control_collection x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1594292148898889728
[junit4] 2> 683052 INFO (coreZkRegister-299-thread-1-processing-n:127.0.0.1:45189_r%2Fng x:collection1 c:control_collection) [n:127.0.0.1:45189_r%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 683052 INFO (coreZkRegister-299-thread-1-processing-n:127.0.0.1:45189_r%2Fng x:collection1 c:control_collection) [n:127.0.0.1:45189_r%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 683052 INFO (coreZkRegister-299-thread-1-processing-n:127.0.0.1:45189_r%2Fng x:collection1 c:control_collection) [n:127.0.0.1:45189_r%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:45189/r/ng/collection1/
[junit4] 2> 683052 INFO (coreZkRegister-299-thread-1-processing-n:127.0.0.1:45189_r%2Fng x:collection1 c:control_collection) [n:127.0.0.1:45189_r%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 683052 INFO (coreZkRegister-299-thread-1-processing-n:127.0.0.1:45189_r%2Fng x:collection1 c:control_collection) [n:127.0.0.1:45189_r%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:45189/r/ng/collection1/ has no replicas
[junit4] 2> 683053 INFO (coreZkRegister-299-thread-1-processing-n:127.0.0.1:45189_r%2Fng x:collection1 c:control_collection) [n:127.0.0.1:45189_r%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 683106 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 683203 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37866/solr ready
[junit4] 2> 683203 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 683226 INFO (coreZkRegister-299-thread-1-processing-n:127.0.0.1:45189_r%2Fng x:collection1 c:control_collection) [n:127.0.0.1:45189_r%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:45189/r/ng/collection1/ shard1
[junit4] 2> 683386 INFO (coreZkRegister-299-thread-1-processing-n:127.0.0.1:45189_r%2Fng x:collection1 c:control_collection) [n:127.0.0.1:45189_r%2Fng c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 684278 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_DF5C6277AD21161E-001/shard-1-001/cores/collection1
[junit4] 2> 684279 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_DF5C6277AD21161E-001/shard-1-001
[junit4] 2> 684306 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 684316 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6da0a504{/r/ng,null,AVAILABLE}
[junit4] 2> 684316 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2014d012{HTTP/1.1,[http/1.1]}{127.0.0.1:40687}
[junit4] 2> 684316 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.e.j.s.Server Started @696285ms
[junit4] 2> 684316 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://lucene2-us-west.apache.org:41587/hdfs__lucene2-us-west.apache.org_41587__home_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-6.6_checkout_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsUnloadDistributedZkTest_DF5C6277AD21161E-001_tempDir-002_jetty1, solrconfig=solrconfig.xml, hostContext=/r/ng, hostPort=40687, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_DF5C6277AD21161E-001/shard-1-001/cores}
[junit4] 2> 684317 ERROR (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 684430 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.6.4
[junit4] 2> 684430 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 684430 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 684430 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-03-07T15:11:17.027Z
[junit4] 2> 684508 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 684508 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_DF5C6277AD21161E-001/shard-1-001/solr.xml
[junit4] 2> 684641 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 684719 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37866/solr
[junit4] 2> 685178 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [n:127.0.0.1:40687_r%2Fng ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 685180 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [n:127.0.0.1:40687_r%2Fng ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 685213 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [n:127.0.0.1:40687_r%2Fng ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40687_r%2Fng
[junit4] 2> 685417 INFO (zkCallback-158-thread-2-processing-n:127.0.0.1:45189_r%2Fng) [n:127.0.0.1:45189_r%2Fng ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 685559 INFO (zkCallback-167-thread-1-processing-n:127.0.0.1:40687_r%2Fng) [n:127.0.0.1:40687_r%2Fng ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 685417 INFO (zkCallback-162-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 685919 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [n:127.0.0.1:40687_r%2Fng ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_DF5C6277AD21161E-001/shard-1-001/cores
[junit4] 2> 685919 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[DF5C6277AD21161E]) [n:127.0.0.1:40687_r%2Fng ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 685926 INFO (coreLoadExecutor-317-thread-1-processing-n:127.0.0.1:40687_r%2Fng) [n:127.0.0.1:40687_r%2Fng ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 685955 INFO (OverseerStateUpdate-99643258690797572-127.0.0.1:45189_r%2Fng-n_0000000000) [n:127.0.0.1:45189_r%2Fng ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 686047 INFO (coreLoadExecutor-317-thread-1-processing-n:127.0.0.1:40687_r%2Fng) [n:127.0.0.1:40687_r%2Fng c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.4
[junit4] 2> 686142 INFO (coreLoadExecutor-317-thread-1-processing-n:127.0.0.1:40687_r%2Fng) [n:127.0.0.1:40687_r%2Fng c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 688505 INFO (coreLoadExecutor-317-thread-1-processing-n:127.0.0.1:40687_r%2Fng) [n:127.0.0.1:40687_r%2Fng c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 688580 INFO (coreLoadExecutor-317-thread-1-processing-n:127.0.0.1:40687_r%2Fng) [n:127.0.0.1:40687_r%2Fng c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1, trusted=true
[junit4] 2> 688580 INFO (coreLoadExecutor-317-thread-1-processing-n:127.0.0.1:40687_r%2Fng) [n:127.0.0.1:40687_r%2Fng c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://lucene2-us-west.apache.org:41587/solr_hdfs_home
[junit4] 2> 688580 INFO (coreLoadExecutor-317-thread-1-processing-n:127.0.0.1:40687_r%2Fng) [n:127.0.0.1:40687_r%2Fng c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 688580 INFO (coreLoadExecutor-317-thread-1-processing-n:127.0.0.1:40687_r%2Fng) [n:127.0.0.1:40687_r%2Fng c:collection1 x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 688580 INFO (coreLoadExecutor-317-thread-1-processing-n:127.0.0.1:40687_r%2Fng) [n:127.0.0.1:40687_r%2Fng c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_DF5C6277AD21161E-001/shard-1-001/cores/collection1], dataDir=[hdfs://lucene2-us-west.apache.org:41587/solr_hdfs_home/collection1/core_node1/data/]
[junit4] 2> 688581 INFO (coreLoadExecutor-317-thread-1-processing-n:127.0.0.1:40687_r%2Fng) [n:127.0.0.1:40687_r%2Fng c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1397dcb1
[junit4] 2> 688581 INFO (coreLoadExecutor-317-thread-1-processing-n:127.0.0.1:40687_r%2Fng) [n:127.0.0.1:40687_r%2Fng c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:41587/solr_hdfs_home/collection1/core_node1/data/snapshot_metadata
[junit4] 2> 688737 INFO (coreLoadExecutor-317-thread-1-processing-n:127.0.0.1:40687_r%2Fng) [n:127.0.0.1:40687_r%2Fng c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 688737 INFO (coreLoadExecutor-317-thread-1-processing-n:127.0.0.1:40687_r%2Fng) [n:127.0.0.1:40687_r%2Fng c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 688737 INFO (coreLoadExecutor-317-thread-1-processing-n:127.0.0.1:40687_r%2Fng) [n:127.0.0.1:40687_r%2Fng c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 689279 INFO (coreLoadExecutor-317-thread-1-processing-n:127.0.0.1:40687_r%2Fng) [n:127.0.0.1:40687_r%2Fng c:collection1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 689279 INFO (coreLoadExecutor-317-thread-1-processing-n:127.0.0.1:40687_r%2Fng) [n:127.0.0.1:40687_r%2Fng c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:41587/solr_hdfs_home/collection1/core_node1/data
[junit4] 2> 689516 INFO (coreLoadExecutor-317-thread-1-processing-n:127.0.0.1:40687_r%2Fng) [n:127.0.0.1:40687_r%2Fng c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:41587/solr_hdfs_home/collection1/core_node1/data/index
[junit4] 2> 689617 INFO (coreLoadExecutor-317-thread-1-processing-n:127.0.0.1:40687_r%2Fng) [n:127.0.0.1:40687_r%2Fng c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 689617 INFO (coreLoadExecutor-317-thread-1-processing-n:127.0.0.1:40687_r%2Fng) [n:127.0.0.1:40687_r%2Fng c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 689617 INFO (coreLoadExecutor-317-thread-1-processing-n:127.0.0.1:40687_r%2Fng) [n:127.0.0.1:40687_r%2Fng c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 689697 INFO (coreLoadExecutor-317-thread-1-processing-n:127.0.0.1:40687_r%2Fng) [n:127.0.0.1:40687_r%2Fng c:collection1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 689697 INFO (coreLoadExecutor-317-thread-1-processing-n:127.0.0.1:40687_r%2Fng) [n:127.0.0.1:40687_r%2Fng c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=43, maxMergeAtOnceExplicit=29, maxMergedSegmentMB=63.90234375, floorSegmentMB=0.830078125, forceMergeDeletesPctAllowed=3.5752316798021857, segmentsPerTier=25.0, maxCFSSegmentSizeMB=1.09375, noCFSRatio=0.7147959364477267
[junit4] 2> 689986 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:43167 is added to blk_1073741826_1002{UCState=COMMITTED, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-b1adebb0-2962-402e-bba2-3aa07a7764fc:NORMAL:127.0.0.1:34423|RBW], ReplicaUC[[DISK]DS-c9a73587-50f4-419c-8404-b3969f878909:NORMAL:127.0.0.1:43167|RBW]]} size 71
[junit4] 2> 690004 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:34423 is added to blk_1073741826_1002 size 71
[junit4] 2> 690477 WARN (coreLoadExecutor-317-thread-1-processing-n:127.0.0.1:40687_r%2Fng) [n:127.0.0.1:40687_r%2Fng c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 690802 INFO (coreLoadExecutor-317-thread-1-processing-n:127.0.0.1:40687_r%2Fng) [n:127.0.0.1:40687_r%2Fng c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 690802 INFO (coreLoadExecutor-317-thread-1-processing-n:127.0.0.1:40687_r%2Fng) [n:127.0.0.1:40687_r%2Fng c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 690822 INFO (coreLoadExecutor-317-thread-1-processing-n:127.0.0.1:40687_r%2Fng) [n:127.0.0.1:40687_r%2Fng c:collection1 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 690930 INFO (coreLoadExecutor-317-thread-1-processing-n:127.0.0.1:40687_r%2Fng) [n:127.0.0.1:40687_r%2Fng c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 690930 INFO (coreLoadExecutor-317-thread-1-processing-n:127.0.0.1:40687_r%2Fng) [n:127.0.0.1:40687_r%2Fng c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 690966 INFO (coreLoadExecutor-317-thread-1-processing-n:127.0.0.1:40687_r%2Fng) [n:127.0.0.1:40687_r%2Fng c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@417990e6
[junit4] 2> 691160 INFO (coreLoadExecutor-317-thread-1-processing-n:127.0.0.1:40687_r%2Fng) [n:127.0.0.1:40687_r%2Fng c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@fe2aa9a[collection1] main]
[junit4] 2> 691161 INFO (coreLoadExecutor-317-thread-1-processing-n:127.0.0.1:40687_r%2Fng) [n:127.0.0.1:40687_r%2Fng c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 691161 INFO (coreLoadExecutor-317-thread-1-processing-n:127.0.0.1:40687_r%2Fng) [n:127.0.0.1:40687_r%2Fng c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 691161 INFO (coreLoadExecutor-317-thread-1-processing-n:127.0.0.1:40687_r%2Fng) [n:127.0.0.1:40687_r%2Fng c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 691208 INFO (searcherExecutor-318-thread-1-processing-n:127.0.0.1:40687_r%2Fng x:collection1 c:collection1) [n:127.0.0.1:40687_r%2Fng c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@fe2aa9a[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 691283 INFO (coreLoadExecutor-317-thread-1-processing-n:127.0.0.1:40687_r%2Fng) [n:127.0.0.1:40687_r%2Fng c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1594292157944954880
[junit4] 2> 691436 INFO (coreZkRegister-312-thread-1-processing-n:127.0.0.1:40687_r%2Fng x:collection1 c:collection1) [n:127.0.0.1:40687_r%2Fng c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 691436 INFO (coreZkRegister-312-thread-1-processing-n:127.0.0.1:40687_r%2Fng x:collection1 c:collection1) [n:127.0.0.1:40687_r%2Fng c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 691436 INFO (coreZkRegister-312-thread-1-processing-n:127.0.0.1:40687_r%2Fng x:collection1 c:collection1) [n:127.0.0.1:40687_r%2Fng c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:40687/r/ng/collection1/
[junit4] 2> 691436 INFO (coreZkRegister-312-thread-1-processing-n:127.0.0.1:40687_r%2Fng x:collection1 c:collection1) [n:127.0.0.1:40687_r%2Fng c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 691436 INFO (coreZkRegister-312-thread-1-processing-n:127.0.0.1:40687_r%2Fng x:collection1 c:collection1) [n:127.0.0.1:40687_r%2Fng c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:40687/r/ng/collection1/ has no replicas
[junit4] 2> 691436 INFO (coreZkRegister-312-thread-1-processing-n:127.0.0.1:40687_r%2Fng x:collection1 c:collection1) [n:127.0.0.1:40687_r%2Fng c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 691439 INFO (coreZkRegister-312-thread-1-processing-n:127.0.0.1:40687_r%2Fng x:collection1 c:collection1) [n:127.0.0.1:40687_r%2Fng c:collection1 s:shard2 r:core_node1 x:co
[...truncated too long message...]
n.java:244)
[junit4] 2> com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:355)
[junit4] 2> com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:13)
[junit4] 2>
[junit4] 2> LeaseRenewer:jenkins@lucene2-us-west.apache.org:41032:
[junit4] 2> java.lang.Thread.sleep(Native Method)
[junit4] 2> org.apache.hadoop.hdfs.LeaseRenewer.run(LeaseRenewer.java:444)
[junit4] 2> org.apache.hadoop.hdfs.LeaseRenewer.access$700(LeaseRenewer.java:71)
[junit4] 2> org.apache.hadoop.hdfs.LeaseRenewer$1.run(LeaseRenewer.java:304)
[junit4] 2> java.lang.Thread.run(Thread.java:748)
[junit4] 2>
[junit4] 2> process reaper:
[junit4] 2> [...sun.*]
[junit4] 2> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] 2> java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] 2> java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] 2> java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] 2> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
[junit4] 2> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] 2> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> java.lang.Thread.run(Thread.java:748)
[junit4] 2>
[junit4] 2> TimeLimitedCollector timer thread:
[junit4] 2> java.lang.Thread.sleep(Native Method)
[junit4] 2> org.apache.lucene.search.TimeLimitingCollector$TimerThread.run(TimeLimitingCollector.java:261)
[junit4] 2>
[junit4] 2> IPC Parameter Sending Thread #7:
[junit4] 2> [...sun.*]
[junit4] 2> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] 2> java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] 2> java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] 2> java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] 2> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
[junit4] 2> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] 2> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> java.lang.Thread.run(Thread.java:748)
[junit4] 2>
[junit4] 2> IPC Parameter Sending Thread #8:
[junit4] 2> [...sun.*]
[junit4] 2> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] 2> java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] 2> java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] 2> java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] 2> java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
[junit4] 2> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] 2> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> java.lang.Thread.run(Thread.java:748)
[junit4] 2>
[junit4] 2> 7847832 INFO (SUITE-HdfsBasicDistributedZkTest-seed#[DF5C6277AD21161E]-worker) [ ] o.a.s.SolrTestCaseJ4 ------------------------------------------------------- Done waiting for tracked resources to be released
[junit4] 2> NOTE: test params are: codec=Lucene62, sim=RandomSimilarity(queryNorm=false,coord=yes): {}, locale=sv-SE, timezone=Europe/Rome
[junit4] 2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=2,free=208046768,total=534249472
[junit4] 2> NOTE: All tests run in this JVM: [TestComplexPhraseLeadingWildcard, TestScoreJoinQPScore, MultiThreadedOCPTest, TestDynamicFieldCollectionResource, TestCloudPseudoReturnFields, ChaosMonkeyNothingIsSafeTest, VMParamsZkACLAndCredentialsProvidersTest, SuggestComponentContextFilterQueryTest, TestStressRecovery, DocumentAnalysisRequestHandlerTest, QueryElevationComponentTest, TestOmitPositions, BasicZkTest, SynonymTokenizerTest, DirectoryFactoryTest, SolrRequestParserTest, PreAnalyzedUpdateProcessorTest, TestBulkSchemaAPI, TestSha256AuthenticationProvider, DistributedFacetPivotWhiteBoxTest, TestExactSharedStatsCache, AnalyticsMergeStrategyTest, TestAuthorizationFramework, TestSolr4Spatial, SolrCoreTest, TestLazyCores, TestXmlQParser, RollingRestartTest, TestFieldCacheSortRandom, TestJsonFacets, RegexBoostProcessorTest, BlockDirectoryTest, TestTolerantUpdateProcessorRandomCloud, TestSortByMinMaxFunction, ScriptEngineTest, ReturnFieldsTest, AliasIntegrationTest, TestStressInPlaceUpdates, TestSchemaVersionResource, TestSolrXml, PeerSyncTest, TestRandomDVFaceting, DeleteNodeTest, TestSolrCloudWithKerberosAlt, TestPathTrie, SolrXmlInZkTest, DistribJoinFromCollectionTest, TestNonDefinedSimilarityFactory, SimpleCollectionCreateDeleteTest, TestSubQueryTransformer, TestExpandComponent, TestSolr4Spatial2, TestSolrQueryParserResource, V2ApiIntegrationTest, TestCSVLoader, NotRequiredUniqueKeyTest, DistributedSpellCheckComponentTest, ExternalFileFieldSortTest, TestSlowCompositeReaderWrapper, DistributedQueryElevationComponentTest, TestBadConfig, SolrGangliaReporterTest, DeleteStatusTest, TestCloudInspectUtil, SortSpecParsingTest, TestReRankQParserPlugin, DistributedExpandComponentTest, HdfsChaosMonkeyNothingIsSafeTest, TestJavabinTupleStreamParser, ChangedSchemaMergeTest, ClusterStateTest, MigrateRouteKeyTest, TestHalfAndHalfDocValues, TestRandomFaceting, TestCloudNestedDocsSort, EnumFieldTest, TestRequestStatusCollectionAPI, SignatureUpdateProcessorFactoryTest, LargeFieldTest, TestDocSet, TestSchemalessBufferedUpdates, DocValuesNotIndexedTest, TestRealTimeGet, TestJmxIntegration, CoreMergeIndexesAdminHandlerTest, TestStressLucene, CustomCollectionTest, SpatialRPTFieldTypeTest, TestFieldResource, TestDistributedSearch, HdfsTlogReplayBufferedWhileIndexingTest, TestFaceting, TestBinaryResponseWriter, TestFieldSortValues, TestMacroExpander, SyncSliceTest, SubstringBytesRefFilterTest, HdfsNNFailoverTest, QueryEqualityTest, TestFreeTextSuggestions, CacheHeaderTest, PropertiesRequestHandlerTest, TestUninvertingReader, TestSearcherReuse, TestQueryWrapperFilter, DefaultValueUpdateProcessorTest, PolyFieldTest, JavabinLoaderTest, TestManagedSynonymFilterFactory, BlockJoinFacetRandomTest, TestCodecSupport, UniqFieldsUpdateProcessorFactoryTest, TestFoldingMultitermQuery, TestStandardQParsers, UUIDFieldTest, BasicDistributedZkTest, ChaosMonkeySafeLeaderTest, UnloadDistributedZkTest, CollectionsAPIDistributedZkTest, LeaderElectionIntegrationTest, TestReplicationHandler, SolrCLIZkUtilsTest, TestCloudJSONFacetJoinDomain, TestConfigSetsAPIZkFailure, TestDistribDocBasedVersion, TestDownShardTolerantSearch, TestLeaderInitiatedRecoveryThread, TestLockTree, TestMiniSolrCloudCluster, TestRebalanceLeaders, TestRequestForwarding, TestSizeLimitedDistributedMap, HdfsBasicDistributedZkTest]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=HdfsBasicDistributedZkTest -Dtests.seed=DF5C6277AD21161E -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/test-data/enwiki.random.lines.txt -Dtests.locale=sv-SE -Dtests.timezone=Europe/Rome -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 0.00s J0 | HdfsBasicDistributedZkTest (suite) <<<
[junit4] > Throwable #1: java.lang.AssertionError: ObjectTracker found 1 object(s) that were not released!!! [NRTCachingDirectory]
[junit4] > org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory
[junit4] > at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
[junit4] > at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
[junit4] > at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:353)
[junit4] > at org.apache.solr.core.SolrCore.cleanupOldIndexDirectories(SolrCore.java:3044)
[junit4] > at org.apache.solr.core.SolrCore.close(SolrCore.java:1575)
[junit4] > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:969)
[junit4] > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:830)
[junit4] > at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:950)
[junit4] > at org.apache.solr.core.CoreContainer.lambda$load$5(CoreContainer.java:565)
[junit4] > at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
[junit4] > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] > at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([DF5C6277AD21161E]:0)
[junit4] > at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:302)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] Completed [689/714 (5!)] on J0 in 939.02s, 1 test, 1 failure <<< FAILURES!
[...truncated 46399 lines...]
[JENKINS] Lucene-Solr-NightlyTests-6.6 - Build # 36 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.6/36/
5 tests failed.
FAILED: org.apache.lucene.search.join.TestCheckJoinIndex.testOrphans
Error Message:
Invalid index
Stack Trace:
java.lang.AssertionError: Invalid index
at __randomizedtesting.SeedInfo.seed([F6A8C9D5BA982C0A:993789BFDA55BC27]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.lucene.search.join.TestCheckJoinIndex.testOrphans(TestCheckJoinIndex.java:92)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.hdfs.HdfsUnloadDistributedZkTest.test
Error Message:
Captured an uncaught exception in thread: Thread[id=3186, name=testExecutor-1568-thread-5, state=RUNNABLE, group=TGRP-HdfsUnloadDistributedZkTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=3186, name=testExecutor-1568-thread-5, state=RUNNABLE, group=TGRP-HdfsUnloadDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:40347: Error CREATEing SolrCore 'multiunload4': Unable to create core [multiunload4] Caused by: Direct buffer memory
at __randomizedtesting.SeedInfo.seed([D4AAAF8E12BF4EE2]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:612)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at org.apache.solr.cloud.BasicDistributedZkTest.lambda$createCollectionInOneInstance$0(BasicDistributedZkTest.java:589)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.core.TestCustomStream.testDynamicLoadingCustomStream
Error Message:
KeeperErrorCode = Session expired for /clusterstate.json
Stack Trace:
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /clusterstate.json
at __randomizedtesting.SeedInfo.seed([D4AAAF8E12BF4EE2:A41FA6ACE8450555]:0)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1212)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:357)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:354)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:354)
at org.apache.solr.common.cloud.ZkStateReader.refreshLegacyClusterState(ZkStateReader.java:532)
at org.apache.solr.common.cloud.ZkStateReader.forceUpdateCollection(ZkStateReader.java:300)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.updateMappingsFromZk(AbstractFullDistribZkTestBase.java:622)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.updateMappingsFromZk(AbstractFullDistribZkTestBase.java:617)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createJettys(AbstractFullDistribZkTestBase.java:430)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:331)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:990)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.client.solrj.TestSolrJErrorHandling.testWithXml
Error Message:
expected:<568> but was:<558>
Stack Trace:
java.lang.AssertionError: expected:<568> but was:<558>
at __randomizedtesting.SeedInfo.seed([EA4D8BD788B670D5:41AF32210AB069B8]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.client.solrj.TestSolrJErrorHandling.doThreads(TestSolrJErrorHandling.java:185)
at org.apache.solr.client.solrj.TestSolrJErrorHandling.doIt(TestSolrJErrorHandling.java:200)
at org.apache.solr.client.solrj.TestSolrJErrorHandling.testWithXml(TestSolrJErrorHandling.java:110)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.hdfs.StressHdfsTest.test
Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:36507/_jej
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:36507/_jej
at __randomizedtesting.SeedInfo.seed([D4AAAF8E12BF4EE2:5CFE9054BC43231A]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:623)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:447)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:388)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1383)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1134)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1073)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at org.apache.solr.cloud.hdfs.StressHdfsTest.createAndDeleteCollection(StressHdfsTest.java:220)
at org.apache.solr.cloud.hdfs.StressHdfsTest.test(StressHdfsTest.java:103)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:515)
... 51 more
Build Log:
[...truncated 8351 lines...]
[junit4] Suite: org.apache.lucene.search.join.TestCheckJoinIndex
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestCheckJoinIndex -Dtests.method=testOrphans -Dtests.seed=F6A8C9D5BA982C0A -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/test-data/enwiki.random.lines.txt -Dtests.locale=sr-ME -Dtests.timezone=America/Winnipeg -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] FAILURE 1.03s J0 | TestCheckJoinIndex.testOrphans <<<
[junit4] > Throwable #1: java.lang.AssertionError: Invalid index
[junit4] > at __randomizedtesting.SeedInfo.seed([F6A8C9D5BA982C0A:993789BFDA55BC27]:0)
[junit4] > at org.apache.lucene.search.join.TestCheckJoinIndex.testOrphans(TestCheckJoinIndex.java:92)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: test params are: codec=FastDecompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=5, maxDocsPerChunk=7, blockSize=9), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=5, blockSize=9)), sim=RandomSimilarity(queryNorm=true,coord=yes): {}, locale=sr-ME, timezone=America/Winnipeg
[junit4] 2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=1,free=208828144,total=253231104
[junit4] 2> NOTE: All tests run in this JVM: [TestCheckJoinIndex]
[junit4] Completed [1/7 (1!)] on J0 in 5.25s, 3 tests, 1 failure <<< FAILURES!
[...truncated 4009 lines...]
[junit4] Suite: org.apache.solr.cloud.hdfs.HdfsUnloadDistributedZkTest
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_D4AAAF8E12BF4EE2-001/init-core-data-001
[junit4] 2> 580509 WARN (SUITE-HdfsUnloadDistributedZkTest-seed#[D4AAAF8E12BF4EE2]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=2 numCloses=2
[junit4] 2> 580509 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[D4AAAF8E12BF4EE2]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields
[junit4] 2> 580510 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[D4AAAF8E12BF4EE2]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
[junit4] 2> 580543 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[D4AAAF8E12BF4EE2]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 2> 583294 WARN (SUITE-HdfsUnloadDistributedZkTest-seed#[D4AAAF8E12BF4EE2]-worker) [ ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 586914 WARN (SUITE-HdfsUnloadDistributedZkTest-seed#[D4AAAF8E12BF4EE2]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 587676 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[D4AAAF8E12BF4EE2]-worker) [ ] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
[junit4] 2> 587776 WARN (SUITE-HdfsUnloadDistributedZkTest-seed#[D4AAAF8E12BF4EE2]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 589261 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[D4AAAF8E12BF4EE2]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 589502 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[D4AAAF8E12BF4EE2]-worker) [ ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/hdfs to ./temp/Jetty_lucene2.us.west_apache_org_38587_hdfs____8qrt21/webapp
[junit4] 2> 591664 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[D4AAAF8E12BF4EE2]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@lucene2-us-west.apache.org:38587
[junit4] 2> 596727 WARN (SUITE-HdfsUnloadDistributedZkTest-seed#[D4AAAF8E12BF4EE2]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 596749 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[D4AAAF8E12BF4EE2]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 596815 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[D4AAAF8E12BF4EE2]-worker) [ ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_43962_datanode____a9lcjh/webapp
[junit4] 2> 597815 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[D4AAAF8E12BF4EE2]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43962
[junit4] 2> 600735 WARN (SUITE-HdfsUnloadDistributedZkTest-seed#[D4AAAF8E12BF4EE2]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 600736 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[D4AAAF8E12BF4EE2]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 601013 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[D4AAAF8E12BF4EE2]-worker) [ ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_45890_datanode____o3lyj9/webapp
[junit4] 2> 606338 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[D4AAAF8E12BF4EE2]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45890
[junit4] 2> 611052 ERROR (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_D4AAAF8E12BF4EE2-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_D4AAAF8E12BF4EE2-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:46485) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 611035 ERROR (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_D4AAAF8E12BF4EE2-001/tempDir-001/hdfsBaseDir/data/data3/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_D4AAAF8E12BF4EE2-001/tempDir-001/hdfsBaseDir/data/data4/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:46485) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 612621 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x97b8f8deaae49: from storage DS-eb10a6dd-9a00-462c-adf9-f20dd959ebae node DatanodeRegistration(127.0.0.1:33171, datanodeUuid=269e155a-c845-48b9-8fe3-a47c91de5130, infoPort=44041, infoSecurePort=0, ipcPort=43728, storageInfo=lv=-56;cid=testClusterID;nsid=393357738;c=0), blocks: 0, hasStaleStorage: true, processing time: 27 msecs
[junit4] 2> 612621 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x97b8f8ceec238: from storage DS-de8360fd-e272-4456-9a38-31229d37fb24 node DatanodeRegistration(127.0.0.1:39183, datanodeUuid=f345b3d7-26c3-4e93-b54c-8bf7541725cb, infoPort=41505, infoSecurePort=0, ipcPort=37452, storageInfo=lv=-56;cid=testClusterID;nsid=393357738;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 612621 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x97b8f8deaae49: from storage DS-5f88f666-4108-4360-9878-fd87a63591d1 node DatanodeRegistration(127.0.0.1:33171, datanodeUuid=269e155a-c845-48b9-8fe3-a47c91de5130, infoPort=44041, infoSecurePort=0, ipcPort=43728, storageInfo=lv=-56;cid=testClusterID;nsid=393357738;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 612645 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x97b8f8ceec238: from storage DS-27955c67-62e1-456f-9abe-a1b0847619be node DatanodeRegistration(127.0.0.1:39183, datanodeUuid=f345b3d7-26c3-4e93-b54c-8bf7541725cb, infoPort=41505, infoSecurePort=0, ipcPort=37452, storageInfo=lv=-56;cid=testClusterID;nsid=393357738;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 614264 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 614274 INFO (Thread-790) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 614274 INFO (Thread-790) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 614276 ERROR (Thread-790) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 614374 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.c.ZkTestServer start zk server on port:35651
[junit4] 2> 614512 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 614523 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 614525 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 614526 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 614544 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 614545 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 614558 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 614560 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 614571 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 614572 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 614573 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 615051 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_D4AAAF8E12BF4EE2-001/control-001/cores/collection1
[junit4] 2> 615069 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 615140 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@574e666a{/,null,AVAILABLE}
[junit4] 2> 615141 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@466a71d1{HTTP/1.1,[http/1.1]}{127.0.0.1:33299}
[junit4] 2> 615141 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.e.j.s.Server Started @626698ms
[junit4] 2> 615141 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://lucene2-us-west.apache.org:46485/hdfs__lucene2-us-west.apache.org_46485__home_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-6.6_checkout_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsUnloadDistributedZkTest_D4AAAF8E12BF4EE2-001_tempDir-002_control_data, hostContext=/, hostPort=33299, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_D4AAAF8E12BF4EE2-001/control-001/cores}
[junit4] 2> 615141 ERROR (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 615141 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.6.3
[junit4] 2> 615141 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 615141 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 615141 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-03-06T10:22:13.011Z
[junit4] 2> 615286 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 615286 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_D4AAAF8E12BF4EE2-001/control-001/solr.xml
[junit4] 2> 615323 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 615324 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35651/solr
[junit4] 2> 615799 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [n:127.0.0.1:33299_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 615799 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [n:127.0.0.1:33299_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:33299_
[junit4] 2> 615800 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [n:127.0.0.1:33299_ ] o.a.s.c.Overseer Overseer (id=99636460397854724-127.0.0.1:33299_-n_0000000000) starting
[junit4] 2> 615957 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [n:127.0.0.1:33299_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33299_
[junit4] 2> 616021 INFO (zkCallback-423-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 616249 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [n:127.0.0.1:33299_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_D4AAAF8E12BF4EE2-001/control-001/cores
[junit4] 2> 616249 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [n:127.0.0.1:33299_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 616309 INFO (coreLoadExecutor-1491-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 616395 INFO (OverseerStateUpdate-99636460397854724-127.0.0.1:33299_-n_0000000000) [n:127.0.0.1:33299_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 617991 INFO (coreLoadExecutor-1491-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_ c:control_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.3
[junit4] 2> 618226 INFO (coreLoadExecutor-1491-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_ c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 619007 INFO (coreLoadExecutor-1491-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_ c:control_collection x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 619472 INFO (coreLoadExecutor-1491-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_ c:control_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection, trusted=true
[junit4] 2> 619561 INFO (coreLoadExecutor-1491-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://lucene2-us-west.apache.org:46485/solr_hdfs_home
[junit4] 2> 619562 INFO (coreLoadExecutor-1491-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 619562 INFO (coreLoadExecutor-1491-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_ c:control_collection x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 619562 INFO (coreLoadExecutor-1491-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_ c:control_collection x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_D4AAAF8E12BF4EE2-001/control-001/cores/collection1], dataDir=[hdfs://lucene2-us-west.apache.org:46485/solr_hdfs_home/control_collection/core_node1/data/]
[junit4] 2> 619562 INFO (coreLoadExecutor-1491-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_ c:control_collection x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@62b17bb9
[junit4] 2> 619563 INFO (coreLoadExecutor-1491-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:46485/solr_hdfs_home/control_collection/core_node1/data/snapshot_metadata
[junit4] 2> 619710 INFO (coreLoadExecutor-1491-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 619746 INFO (coreLoadExecutor-1491-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 619746 INFO (coreLoadExecutor-1491-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 623193 INFO (coreLoadExecutor-1491-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_ c:control_collection x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 623205 INFO (coreLoadExecutor-1491-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:46485/solr_hdfs_home/control_collection/core_node1/data
[junit4] 2> 623447 INFO (coreLoadExecutor-1491-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:46485/solr_hdfs_home/control_collection/core_node1/data/index
[junit4] 2> 623503 INFO (coreLoadExecutor-1491-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 623503 INFO (coreLoadExecutor-1491-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 623503 INFO (coreLoadExecutor-1491-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 623555 INFO (coreLoadExecutor-1491-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_ c:control_collection x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 623556 INFO (coreLoadExecutor-1491-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_ c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=7, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 625636 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:39183 is added to blk_1073741825_1001{UCState=COMMITTED, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-de8360fd-e272-4456-9a38-31229d37fb24:NORMAL:127.0.0.1:39183|RBW], ReplicaUC[[DISK]DS-eb10a6dd-9a00-462c-adf9-f20dd959ebae:NORMAL:127.0.0.1:33171|RBW]]} size 71
[junit4] 2> 625689 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:33171 is added to blk_1073741825_1001 size 71
[junit4] 2> 626167 WARN (coreLoadExecutor-1491-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_ c:control_collection x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 626566 INFO (coreLoadExecutor-1491-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_ c:control_collection x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 626566 INFO (coreLoadExecutor-1491-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_ c:control_collection x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 626566 INFO (coreLoadExecutor-1491-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_ c:control_collection x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 626639 INFO (coreLoadExecutor-1491-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_ c:control_collection x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 626639 INFO (coreLoadExecutor-1491-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_ c:control_collection x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 626656 INFO (coreLoadExecutor-1491-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_ c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1593055026, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 626944 INFO (coreLoadExecutor-1491-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_ c:control_collection x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@e032f9b[collection1] main]
[junit4] 2> 626977 INFO (coreLoadExecutor-1491-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_ c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 626977 INFO (coreLoadExecutor-1491-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_ c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 626992 INFO (coreLoadExecutor-1491-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_ c:control_collection x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 626994 INFO (searcherExecutor-1492-thread-1-processing-n:127.0.0.1:33299_ x:collection1 c:control_collection) [n:127.0.0.1:33299_ c:control_collection x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@e032f9b[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 627015 INFO (coreLoadExecutor-1491-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_ c:control_collection x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1594183379724533760
[junit4] 2> 627036 INFO (coreZkRegister-1484-thread-1-processing-n:127.0.0.1:33299_ x:collection1 c:control_collection) [n:127.0.0.1:33299_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 627036 INFO (coreZkRegister-1484-thread-1-processing-n:127.0.0.1:33299_ x:collection1 c:control_collection) [n:127.0.0.1:33299_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 627036 INFO (coreZkRegister-1484-thread-1-processing-n:127.0.0.1:33299_ x:collection1 c:control_collection) [n:127.0.0.1:33299_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:33299/collection1/
[junit4] 2> 627036 INFO (coreZkRegister-1484-thread-1-processing-n:127.0.0.1:33299_ x:collection1 c:control_collection) [n:127.0.0.1:33299_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 627036 INFO (coreZkRegister-1484-thread-1-processing-n:127.0.0.1:33299_ x:collection1 c:control_collection) [n:127.0.0.1:33299_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:33299/collection1/ has no replicas
[junit4] 2> 627036 INFO (coreZkRegister-1484-thread-1-processing-n:127.0.0.1:33299_ x:collection1 c:control_collection) [n:127.0.0.1:33299_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 627161 INFO (coreZkRegister-1484-thread-1-processing-n:127.0.0.1:33299_ x:collection1 c:control_collection) [n:127.0.0.1:33299_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:33299/collection1/ shard1
[junit4] 2> 627162 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 627189 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:35651/solr ready
[junit4] 2> 627189 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 627322 INFO (coreZkRegister-1484-thread-1-processing-n:127.0.0.1:33299_ x:collection1 c:control_collection) [n:127.0.0.1:33299_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 627748 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_D4AAAF8E12BF4EE2-001/shard-1-001/cores/collection1
[junit4] 2> 627748 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_D4AAAF8E12BF4EE2-001/shard-1-001
[junit4] 2> 627750 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 627751 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2ff4b313{/,null,AVAILABLE}
[junit4] 2> 627751 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@eafc2ed{HTTP/1.1,[http/1.1]}{127.0.0.1:40347}
[junit4] 2> 627760 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.e.j.s.Server Started @639310ms
[junit4] 2> 627760 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://lucene2-us-west.apache.org:46485/hdfs__lucene2-us-west.apache.org_46485__home_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-6.6_checkout_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsUnloadDistributedZkTest_D4AAAF8E12BF4EE2-001_tempDir-002_jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=40347, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_D4AAAF8E12BF4EE2-001/shard-1-001/cores}
[junit4] 2> 627760 ERROR (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 627760 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.6.3
[junit4] 2> 627760 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 627760 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 627760 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-03-06T10:22:25.630Z
[junit4] 2> 627779 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 627779 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_D4AAAF8E12BF4EE2-001/shard-1-001/solr.xml
[junit4] 2> 627805 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 627806 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35651/solr
[junit4] 2> 627992 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [n:127.0.0.1:40347_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 627998 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [n:127.0.0.1:40347_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 628041 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [n:127.0.0.1:40347_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40347_
[junit4] 2> 628162 INFO (zkCallback-432-thread-1-processing-n:127.0.0.1:40347_) [n:127.0.0.1:40347_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 628163 INFO (zkCallback-423-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 628163 INFO (zkCallback-427-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 628364 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [n:127.0.0.1:40347_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_D4AAAF8E12BF4EE2-001/shard-1-001/cores
[junit4] 2> 628364 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [n:127.0.0.1:40347_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 628411 INFO (coreLoadExecutor-1502-thread-1-processing-n:127.0.0.1:40347_) [n:127.0.0.1:40347_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 628435 INFO (OverseerStateUpdate-99636460397854724-127.0.0.1:33299_-n_0000000000) [n:127.0.0.1:33299_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 629503 INFO (coreLoadExecutor-1502-thread-1-processing-n:127.0.0.1:40347_) [n:127.0.0.1:40347_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.3
[junit4] 2> 629561 INFO (coreLoadExecutor-1502-thread-1-processing-n:127.0.0.1:40347_) [n:127.0.0.1:40347_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 630186 INFO (coreLoadExecutor-1502-thread-1-processing-n:127.0.0.1:40347_) [n:127.0.0.1:40347_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 630397 INFO (coreLoadExecutor-1502-thread-1-processing-n:127.0.0.1:40347_) [n:127.0.0.1:40347_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1, trusted=true
[junit4] 2> 630398 INFO (coreLoadExecutor-1502-thread-1-processing-n:127.0.0.1:40347_) [n:127.0.0.1:40347_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://lucene2-us-west.apache.org:46485/solr_hdfs_home
[junit4] 2> 630435 INFO (coreLoadExecutor-1502-thread-1-processing-n:127.0.0.1:40347_) [n:127.0.0.1:40347_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 630435 INFO (coreLoadExecutor-1502-thread-1-processing-n:127.0.0.1:40347_) [n:127.0.0.1:40347_ c:collection1 x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 630435 INFO (coreLoadExecutor-1502-thread-1-processing-n:127.0.0.1:40347_) [n:127.0.0.1:40347_ c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_D4AAAF8E12BF4EE2-001/shard-1-001/cores/collection1], dataDir=[hdfs://lucene2-us-west.apache.org:46485/solr_hdfs_home/collection1/core_node1/data/]
[junit4] 2> 630435 INFO (coreLoadExecutor-1502-thread-1-processing-n:127.0.0.1:40347_) [n:127.0.0.1:40347_ c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@62b17bb9
[junit4] 2> 630436 INFO (coreLoadExecutor-1502-thread-1-processing-n:127.0.0.1:40347_) [n:127.0.0.1:40347_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:46485/solr_hdfs_home/collection1/core_node1/data/snapshot_metadata
[junit4] 2> 630493 INFO (coreLoadExecutor-1502-thread-1-processing-n:127.0.0.1:40347_) [n:127.0.0.1:40347_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 630493 INFO (coreLoadExecutor-1502-thread-1-processing-n:127.0.0.1:40347_) [n:127.0.0.1:40347_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 630493 INFO (coreLoadExecutor-1502-thread-1-processing-n:127.0.0.1:40347_) [n:127.0.0.1:40347_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 632296 INFO (coreLoadExecutor-1502-thread-1-processing-n:127.0.0.1:40347_) [n:127.0.0.1:40347_ c:collection1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 632297 INFO (coreLoadExecutor-1502-thread-1-processing-n:127.0.0.1:40347_) [n:127.0.0.1:40347_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:46485/solr_hdfs_home/collection1/core_node1/data
[junit4] 2> 632403 INFO (coreLoadExecutor-1502-thread-1-processing-n:127.0.0.1:40347_) [n:127.0.0.1:40347_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:46485/solr_hdfs_home/collection1/core_node1/data/index
[junit4] 2> 632495 INFO (coreLoadExecutor-1502-thread-1-processing-n:127.0.0.1:40347_) [n:127.0.0.1:40347_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 632495 INFO (coreLoadExecutor-1502-thread-1-processing-n:127.0.0.1:40347_) [n:127.0.0.1:40347_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 632495 INFO (coreLoadExecutor-1502-thread-1-processing-n:127.0.0.1:40347_) [n:127.0.0.1:40347_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 633394 INFO (coreLoadExecutor-1502-thread-1-processing-n:127.0.0.1:40347_) [n:127.0.0.1:40347_ c:collection1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 633395 INFO (coreLoadExecutor-1502-thread-1-processing-n:127.0.0.1:40347_) [n:127.0.0.1:40347_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=7, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 633469 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:33171 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-de8360fd-e272-4456-9a38-31229d37fb24:NORMAL:127.0.0.1:39183|RBW], ReplicaUC[[DISK]DS-5f88f666-4108-4360-9878-fd87a63591d1:NORMAL:127.0.0.1:33171|RBW]]} size 0
[junit4] 2> 633482 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:39183 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-de8360fd-e272-4456-9a38-31229d37fb24:NORMAL:127.0.0.1:39183|RBW], ReplicaUC[[DISK]DS-5f88f666-4108-4360-9878-fd87a63591d1:NORMAL:127.0.0.1:33171|RBW]]} size 0
[junit4] 2> 633549 WARN (coreLoadExecutor-1502-thread-1-processing-n:127.0.0.1:40347_) [n:127.0.0.1:40347_ c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 633736 INFO (coreLoadExecutor-1502-thread-1-processing-n:127.0.0.1:40347_) [n:127.0.0.1:40347_ c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 633736 INFO (coreLoadExecutor-1502-thread-1-processing-n:127.0.0.1:40347_) [n:127.0.0.1:40347_ c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 633736 INFO (coreLoadExecutor-1502-thread-1-processing-n:127.0.0.1:40347_) [n:127.0.0.1:40347_ c:collection1 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 633790 INFO (coreLoadExecutor-1502-thread-1-processing-n:127.0.0.1:40347_) [n:127.0.0.1:40347_ c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 633790 INFO (coreLoadExecutor-1502-thread-1-processing-n:127.0.0.1:40347_) [n:127.0.0.1:40347_ c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 633885 INFO (coreLoadExecutor-1502-thread-1-processing-n:127.0.0.1:40347_) [n:127.0.0.1:40347_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1593055026, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 634197 INFO (coreLoadExecutor-1502-thread-1-processing-n:127.0.0.1:40347_) [n:127.0.0.1:40347_ c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3fabac7e[collection1] main]
[junit4] 2> 634283 INFO (coreLoadExecutor-1502-thread-1-processing-n:127.0.0.1:40347_) [n:127.0.0.1:40347_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 634291 INFO (coreLoadExecutor-1502-thread-1-processing-n:127.0.0.1:40347_) [n:127.0.0.1:40347_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 634291 INFO (coreLoadExecutor-1502-thread-1-processing-n:127.0.0.1:40347_) [n:127.0.0.1:40347_ c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 634304 INFO (searcherExecutor-1503-thread-1-processing-n:127.0.0.1:40347_ x:collection1 c:collection1) [n:127.0.0.1:40347_ c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@3fabac7e[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 634420 INFO (coreLoadExecutor-1502-thread-1-processing-n:127.0.0.1:40347_) [n:127.0.0.1:40347_ c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1594183387489239040
[junit4] 2> 634457 INFO (coreZkRegister-1497-thread-1-processing-n:127.0.0.1:40347_ x:collection1 c:collection1) [n:127.0.0.1:40347_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 634457 INFO (coreZkRegister-1497-thread-1-processing-n:127.0.0.1:40347_ x:collection1 c:collection1) [n:127.0.0.1:40347_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 634457 INFO (coreZkRegister-1497-thread-1-processing-n:127.0.0.1:40347_ x:collection1 c:collection1) [n:127.0.0.1:40347_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:40347/collection1/
[junit4] 2> 634457 INFO (coreZkRegister-1497-thread-1-processing-n:127.0.0.1:40347_ x:collection1 c:collection1) [n:127.0.0.1:40347_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 634457 INFO (coreZkRegister-1497-thread-1-processing-n:127.0.0.1:40347_ x:collection1 c:collection1) [n:127.0.0.1:40347_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:40347/collection1/ has no replicas
[junit4] 2> 634457 INFO (coreZkRegister-1497-thread-1-processing-n:127.0.0.1:40347_ x:collection1 c:collection1) [n:127.0.0.1:40347_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 634501 INFO (coreZkRegister-1497-thread-1-processing-n:127.0.0.1:40347_ x:collection1 c:collection1) [n:127.0.0.1:40347_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:40347/collection1/ shard2
[junit4] 2> 634669 INFO (coreZkRegister-1497-thread-1-processing-n:127.0.0.1:40347_ x:collection1 c:collection1) [n:127.0.0.1:40347_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 635595 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_D4AAAF8E12BF4EE2-001/shard-2-001/cores/collection1
[junit4] 2> 635595 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_D4AAAF8E12BF4EE2-001/shard-2-001
[junit4] 2> 635614 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 635635 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6ed800e0{/,null,AVAILABLE}
[junit4] 2> 635635 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@71a74b2{HTTP/1.1,[http/1.1]}{127.0.0.1:33173}
[junit4] 2> 635635 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.e.j.s.Server Started @647193ms
[junit4] 2> 635635 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://lucene2-us-west.apache.org:46485/hdfs__lucene2-us-west.apache.org_46485__home_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-6.6_checkout_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsUnloadDistributedZkTest_D4AAAF8E12BF4EE2-001_tempDir-002_jetty2, solrconfig=solrconfig.xml, hostContext=/, hostPort=33173, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_D4AAAF8E12BF4EE2-001/shard-2-001/cores}
[junit4] 2> 635635 ERROR (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 635635 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.6.3
[junit4] 2> 635635 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 635636 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 635636 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-03-06T10:22:33.506Z
[junit4] 2> 635696 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 635696 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_D4AAAF8E12BF4EE2-001/shard-2-001/solr.xml
[junit4] 2> 635736 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 635736 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35651/solr
[junit4] 2> 636041 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [n:127.0.0.1:33173_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 636043 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [n:127.0.0.1:33173_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 636077 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [n:127.0.0.1:33173_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33173_
[junit4] 2> 636078 INFO (zkCallback-427-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 636079 INFO (zkCallback-438-thread-1-processing-n:127.0.0.1:33173_) [n:127.0.0.1:33173_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 636096 INFO (zkCallback-432-thread-1-processing-n:127.0.0.1:40347_) [n:127.0.0.1:40347_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 636112 INFO (zkCallback-423-thread-1-processing-n:127.0.0.1:33299_) [n:127.0.0.1:33299_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 636880 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [n:127.0.0.1:33173_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_D4AAAF8E12BF4EE2-001/shard-2-001/cores
[junit4] 2> 636881 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[D4AAAF8E12BF4EE2]) [n:127.0.0.1:33173_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 636881 INFO (coreLoadExecutor-1513-thread-1-processing-n:127.0.0.1:33173_) [n:127.0.0.1:33173_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 636966 INFO (OverseerStateUpdate-99636460397854724-127.0.0.1:33299_-n_0000000000) [n:127.0.0.1:33299_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 637969 INFO (coreLoadExecutor-1513-thread-1-processing-n:127.0.0.1:33173_) [n:127.0.0.1:33173_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.3
[junit4] 2> 638018 INFO (coreLoadExecutor-1513-thread-1-processing-n:127.0.0.1:33173_) [n:127.0.0.1:33173_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 638464 INFO (coreLoadExecutor-1513-thread-1-processing-n:127.0.0.1:33173_) [n:127.0.0.1:33173_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 638512 INFO (coreLoadExecutor-1513-thread-1-processing-n:127.0.0.1:33173_) [n:127.0.0.1:33173_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1, trusted=true
[junit4] 2> 638513 INFO (coreLoadExecutor-1513-thread-1-processing-n:127.0.0.1:33173_) [n:127.0.0.1:33173_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://lucene2-us-west.apache.org:46485/solr_hdfs_home
[junit4] 2> 638514 INFO (coreLoadExecutor-1513-thread-1-processing-n:127.0.0.1:33173_) [n:127.0.0.1:33173_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 638514 INFO (coreLoadExecutor-1513-thread-1-processing-n:127.0.0.1:33173_) [n:127.0.0.1:33173_ c:collection1 x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 638522 INFO (coreLoadExecutor-1513-thread-1-processing-n:127.0.0.1:33173_) [n:127.0.0.1:33173_ c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_D4AAAF8E12BF4EE2-001/shard-2-001/cores/collection1], dataDir=[hdfs://lucene2-us-west.apache.org:46485/solr_hdfs_home/collection1/core_node2/data/]
[junit4] 2> 638523 INFO (coreLoadExecutor-1513-thread-1-processing-n:127.0.0.1:33173_) [n:127.0.0.1:33173_ c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@62b17bb9
[junit4] 2> 638523 INFO (coreLoadExecutor-1513-thread-1-processing-n:127.0.0.1:33173_) [n:127.0.0.1:33173_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:46485/solr_hdfs_home/collection1/core_node2/data/snapshot_metadata
[junit4] 2> 638559 INFO (coreLoadExecutor-1513-thread-1-processing-n:127.0.0.1:33173_) [n:127.0.0.1:33173_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 638559 INFO (coreLoadExecutor-1513-thread-1-processing-n:127.0.0.1:33173_) [n:127.0.0.1:33173_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 638559 INFO (coreLoadExecutor-1513-thread-1-processing-n:127.0.0.1:33173_) [n:127.0.0.1:33173_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 638705 INFO (coreLoadExecutor-1513-thread-1-processing-n:127.0.0.1:33173_) [n:127.0.0.1:33173_ c:collection1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 638731 INFO (coreLoadExecutor-1513-thread-1-processing-n:127.0.0.1:33173_) [n:127.0.0.1:33173_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:46485/solr_hdfs_home/collection1/core_node2/data
[junit4] 2> 638790 INFO (coreLoadExecutor-1513-thread-1-processing-n:127.0.0.1:33173_) [n:127.0.0.1:33173_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:46485/solr_hdfs_home/collection1/core_node2/data/index
[junit4] 2> 638796 INFO (coreLoadExecutor-1513-thread-1-processing-n:127.0.0.1:33173_) [n:127.0.0.1:33173_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 638796 INFO (coreLoadExecutor-1513-thread-1-processing-n:127.0.0.1:33173_) [n:127.0.0.1:33173_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 638796 INFO (coreLoadExecutor-1513-thread-1-processing-n:127.0.0.1:33173_) [n:127.0.0.1:33173_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 638862 INFO (coreLoadExecutor-1513-thread-1-processing-n:127.0.0.1:33173_) [n:127.0.0.1:33173_ c:collection1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 638863 INFO (coreLoadExecutor-1513-thread-1-processing-n:127.0.0.1:33173_) [n:127.0.0.1:33173_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=7, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 638941 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:33171 is added to blk_1073741827_1003{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-27955c67-62e1-456f-9abe-a1b0847619be:NORMAL:127.0.0.1:39183|RBW], ReplicaUC[[DISK]DS-eb10a6dd-9a00-462c-adf9-f20dd959ebae:NORMAL:127.0.0.1:33171|FINALIZED]]} size 0
[junit4] 2> 638942 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:39183 is added to blk_1073741827_1003{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-27955c67-62e1-456f-9abe-a1b0847619be:NORMAL:127.0.0.1:39183|RBW], ReplicaUC[[DISK]DS-eb10a6dd-9a00-462c-adf9-f20dd959ebae:NORMAL:127.0.0.1:33171|FINALIZED]]} size 0
[junit4] 2> 638978 WARN (coreLoadExecutor-1513-thread-1-processing-n:127.0.0.1:33173_) [n:127.0.0.1:33173_ c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 639167 INFO (coreLoadExecutor-1513-thread-1-processing-n:127.0.0.1:33173_) [n:127.0.0.1:33173_ c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 639167 INFO (coreLoadExecutor-1513-thread-1-processing-n:127.0.0.1:33173_) [n:127.0.0.1:33173_ c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 639167 INFO (coreLoadExecutor-1513-thread-1-processing-n:127.0.0.1:33173_) [n:127.0.0.1:33173_ c:collection1 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 639269 INFO (coreLoadExecutor-1513-thread-1-processing-n:127.0.0.1:33173_) [n:127.0.0.1:33173_ c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 639269 INFO (coreLoadExecutor-1513-thread-1-processing-n:127.0.0.1:33173_) [n:127.0.0.1:33173_ c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 639343 INFO (coreLoadExecutor-1513-thread-1-processing-n:127.0.0.1:33173_) [n:127.0.0.1:33173_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1593055026, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 639434 INFO (coreLoadExecutor-1513-thread-1-processing-n:127.0.0.1:33173_) [n:127.0.0.1:33173_ c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@70155ef8[collection1] main]
[junit4] 2> 639458 INFO (coreLoadExecutor-1513-thread-1-processing-n:127.0.0.1:33173_) [n:127.0.0.1:33173_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 639458 INFO (coreLoadExecutor-1513-thread-1-processing-n:127.0.0.1:33173_) [n:127.0.0.1:33173_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 639458 INFO (coreLoadExecutor-1513-thread-1-processing-n:127.0.0.1:33173_) [n:127.0.0.1:33173_ c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 639460 INFO (searcherExecutor-1514-thread-1-processing-n:127.0.0.1:33173_ x:collection1 c:collection1) [n:127.0.0.1:33173_ c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@70155ef8[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 639485 INFO (coreLoadExecutor-1513-thread-1-processing-n:127.0.0.1:33173_) [n:127.0.0.1:33173_ c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1594183392800276480
[junit4] 2> 639614 INFO (coreZkRegister-1508-thread-1-processing-n:127.0.0.1:33173_ x:collection1 c:collection1) [n:127.0.0.1:33173_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 639628 INFO (coreZkRegister-1508-thread-1-processing-n:127.0.0.1:33173_ x:collection1 c:collection1) [n:127.0.0.1:33173_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 639628 INFO (coreZkRegister-1508-thread-1-processing-n:127.0.0.1:33173_ x:collection1 c:collection1) [n:127.0.0.1:33173_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:33173/collection1/
[junit4] 2> 639628 INFO (coreZkRegister-1508-thread-1-processing-n:127.0.0.1:33173_ x:collection1 c:collection1) [n:127.0.0.1:33173_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 639628 INFO (coreZkRegister-1508-thread-1-processing-n:127.0.0.1:33173_ x:collection1 c:collection1) [n:127.0.0.1:33173_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:33173/collection1/ has no replicas
[junit4] 2> 639628 INFO (coreZkRegister-1508-thread-1-processing-n:127.0.0.1:33173_ x:collection1 c:collection1) [n:127.0.0.1:33173_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4]
[...truncated too long message...]
llection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-203-thread-1]: start flush: applyAllDeletes=true
[junit4] 2> 221094 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-203-thread-1]: index before flush _6(6.6.3):C392/391:delGen=1
[junit4] 2> 221094 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [DW][coreCloseExecutor-203-thread-1]: startFullFlush
[junit4] 2> 221094 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [DW][coreCloseExecutor-203-thread-1]: coreCloseExecutor-203-thread-1 finishFullFlush success=true
[junit4] 2> 221094 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-203-thread-1]: apply all deletes during flush
[junit4] 2> 221094 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-203-thread-1]: now apply all deletes for all segments maxDoc=392
[junit4] 2> 221094 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [BD][coreCloseExecutor-203-thread-1]: applyDeletes: open segment readers took 0 msec
[junit4] 2> 221095 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [BD][coreCloseExecutor-203-thread-1]: applyDeletes: no segments; skipping
[junit4] 2> 221095 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [BD][coreCloseExecutor-203-thread-1]: prune sis=segments_6: _6(6.6.3):C392/391:delGen=1 minGen=31 packetCount=0
[junit4] 2> 221095 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [MS][coreCloseExecutor-203-thread-1]: updateMergeThreads ioThrottle=true targetMBPerSec=10240.0 MB/sec
[junit4] 2> 221095 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [MS][coreCloseExecutor-203-thread-1]: now merge
[junit4] 2> 221095 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [MS][coreCloseExecutor-203-thread-1]: index: _6(6.6.3):C392/391:delGen=1
[junit4] 2> 221095 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [MS][coreCloseExecutor-203-thread-1]: no more merges pending; now return
[junit4] 2> 221095 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-203-thread-1]: waitForMerges
[junit4] 2> 221095 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-203-thread-1]: waitForMerges done
[junit4] 2> 221095 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-203-thread-1]: commit: start
[junit4] 2> 221095 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-203-thread-1]: commit: enter lock
[junit4] 2> 221095 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-203-thread-1]: commit: now prepare
[junit4] 2> 221095 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-203-thread-1]: prepareCommit: flush
[junit4] 2> 221095 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-203-thread-1]: index before flush _6(6.6.3):C392/391:delGen=1
[junit4] 2> 221095 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [DW][coreCloseExecutor-203-thread-1]: startFullFlush
[junit4] 2> 221095 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-203-thread-1]: apply all deletes during flush
[junit4] 2> 221095 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-203-thread-1]: now apply all deletes for all segments maxDoc=392
[junit4] 2> 221095 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [BD][coreCloseExecutor-203-thread-1]: applyDeletes: open segment readers took 0 msec
[junit4] 2> 221095 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [BD][coreCloseExecutor-203-thread-1]: applyDeletes: no segments; skipping
[junit4] 2> 221095 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [BD][coreCloseExecutor-203-thread-1]: prune sis=segments_6: _6(6.6.3):C392/391:delGen=1 minGen=31 packetCount=0
[junit4] 2> 221095 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [DW][coreCloseExecutor-203-thread-1]: coreCloseExecutor-203-thread-1 finishFullFlush success=true
[junit4] 2> 221095 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-203-thread-1]: startCommit(): start
[junit4] 2> 221095 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-203-thread-1]: skip startCommit(): no changes pending
[junit4] 2> 221095 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreCloseExecutor-203-thread-1]: delete []
[junit4] 2> 221095 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-203-thread-1]: commit: pendingCommit == null; skip
[junit4] 2> 221095 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-203-thread-1]: commit: took 0.3 msec
[junit4] 2> 221095 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-203-thread-1]: commit: done
[junit4] 2> 221095 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-203-thread-1]: rollback
[junit4] 2> 221095 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-203-thread-1]: all running merges have aborted
[junit4] 2> 221095 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-203-thread-1]: rollback: done finish merges
[junit4] 2> 221095 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [DW][coreCloseExecutor-203-thread-1]: abort
[junit4] 2> 221095 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [DW][coreCloseExecutor-203-thread-1]: done abort success=true
[junit4] 2> 221095 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-203-thread-1]: rollback: infos=_6(6.6.3):C392/391:delGen=1
[junit4] 2> 221095 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreCloseExecutor-203-thread-1]: now checkpoint "_6(6.6.3):C392/391:delGen=1" [1 segments ; isCommit = false]
[junit4] 2> 221095 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreCloseExecutor-203-thread-1]: delete []
[junit4] 2> 221095 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreCloseExecutor-203-thread-1]: 0 msec to checkpoint
[junit4] 2> 221095 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreCloseExecutor-203-thread-1]: delete []
[junit4] 2> 221095 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreCloseExecutor-203-thread-1]: delete []
[junit4] 2> 221279 INFO (coreCloseExecutor-203-thread-1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.collection1
[junit4] 2> 221293 INFO (SUITE-TestSolrJErrorHandling-seed#[EA4D8BD788B670D5]-worker) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@3c8ec091{/solr,null,UNAVAILABLE}
[junit4] 2> 221330 INFO (SUITE-TestSolrJErrorHandling-seed#[EA4D8BD788B670D5]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.TestSolrJErrorHandling_EA4D8BD788B670D5-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): {}, docValues:{}, maxPointsInLeafNode=1255, maxMBSortInHeap=5.055302569082642, sim=RandomSimilarity(queryNorm=true,coord=no): {}, locale=sr, timezone=Asia/Baghdad
[junit4] 2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=1,free=117913912,total=345505792
[junit4] 2> NOTE: All tests run in this JVM: [TestDelegationTokenResponse, ConcurrentUpdateSolrClientBuilderTest, LessThanEqualToEvaluatorTest, TestSolrProperties, SelectWithEvaluatorsTest, HyperbolicCosineEvaluatorTest, TestFastInputStream, CommonAdminParamsTest, RoundEvaluatorTest, GetByIdTest, SolrDocumentTest, DocumentAnalysisResponseTest, TestCollectionAdminRequest, SubtractEvaluatorTest, MergeIndexesEmbeddedTest, CoalesceEvaluatorTest, UuidEvaluatorTest, ArcSineEvaluatorTest, SolrExampleStreamingBinaryTest, LBHttpSolrClientBuilderTest, SolrExampleJettyTest, SolrExampleEmbeddedTest, LargeVolumeBinaryJettyTest, SolrParamTest, QueryResponseTest, TestNamedListCodec, TestDocumentObjectBinder, SolrQueryTest, ModifiableSolrParamsTest, TestXMLEscaping, HttpClientUtilTest, TestUpdateRequestCodec, FieldAnalysisResponseTest, ClientUtilsTest, NamedListTest, AnlysisResponseBaseTest, CollectionAdminRequestRequiredParamsTest, TestSolrJErrorHandling]
[junit4] Completed [60/130 (1!)] on J1 in 78.30s, 4 tests, 1 failure <<< FAILURES!
[...truncated 45656 lines...]
[JENKINS] Lucene-Solr-NightlyTests-6.6 - Build # 35 - Still
unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.6/35/
5 tests failed.
FAILED: org.apache.solr.cloud.CdcrReplicationHandlerTest.testReplicationWithBufferedUpdates
Error Message:
Timeout while trying to assert number of documents @ source_collection
Stack Trace:
java.lang.AssertionError: Timeout while trying to assert number of documents @ source_collection
at __randomizedtesting.SeedInfo.seed([D1CF6CAB31D9C539:2C63CB5744A59AE]:0)
at org.apache.solr.cloud.BaseCdcrDistributedZkTest.assertNumDocs(BaseCdcrDistributedZkTest.java:271)
at org.apache.solr.cloud.CdcrReplicationHandlerTest.testReplicationWithBufferedUpdates(CdcrReplicationHandlerTest.java:233)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.AssertionError: expected:<50> but was:<60>
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.BaseCdcrDistributedZkTest.assertNumDocs(BaseCdcrDistributedZkTest.java:262)
... 42 more
FAILED: org.apache.solr.cloud.ShardSplitTest.testSplitAfterFailedSplit
Error Message:
expected:<1> but was:<2>
Stack Trace:
java.lang.AssertionError: expected:<1> but was:<2>
at __randomizedtesting.SeedInfo.seed([D1CF6CAB31D9C539:2882FF040DAC88B3]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.ShardSplitTest.testSplitAfterFailedSplit(ShardSplitTest.java:280)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.ZkSolrClientTest.testMultipleWatchesAsync
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([D1CF6CAB31D9C539:B979BF09A43DC4A7]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.cloud.ZkSolrClientTest.testMultipleWatchesAsync(ZkSolrClientTest.java:257)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.hdfs.StressHdfsTest.test
Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:35670
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:35670
at __randomizedtesting.SeedInfo.seed([D1CF6CAB31D9C539:599B53719F25A8C1]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:623)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:447)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:388)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1383)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1134)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1073)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at org.apache.solr.cloud.hdfs.StressHdfsTest.createAndDeleteCollection(StressHdfsTest.java:220)
at org.apache.solr.cloud.hdfs.StressHdfsTest.test(StressHdfsTest.java:103)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:515)
... 51 more
FAILED: org.apache.solr.core.HdfsDirectoryFactoryTest.testInitArgsOrSysPropConfig
Error Message:
The max direct memory is likely too low. Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~134.217728 MB.
Stack Trace:
java.lang.RuntimeException: The max direct memory is likely too low. Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~134.217728 MB.
at __randomizedtesting.SeedInfo.seed([D1CF6CAB31D9C539:2660A580CC502F12]:0)
at org.apache.solr.core.HdfsDirectoryFactory.createBlockCache(HdfsDirectoryFactory.java:312)
at org.apache.solr.core.HdfsDirectoryFactory.getBlockDirectoryCache(HdfsDirectoryFactory.java:288)
at org.apache.solr.core.HdfsDirectoryFactory.create(HdfsDirectoryFactory.java:228)
at org.apache.solr.core.HdfsDirectoryFactoryTest.testInitArgsOrSysPropConfig(HdfsDirectoryFactoryTest.java:108)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Bits.java:694)
at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
at org.apache.solr.store.blockcache.BlockCache.<init>(BlockCache.java:75)
at org.apache.solr.core.HdfsDirectoryFactory.createBlockCache(HdfsDirectoryFactory.java:310)
... 42 more
Build Log:
[...truncated 12242 lines...]
[junit4] Suite: org.apache.solr.cloud.ZkSolrClientTest
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ZkSolrClientTest_D1CF6CAB31D9C539-001/init-core-data-001
[junit4] 2> 668539 WARN (SUITE-ZkSolrClientTest-seed#[D1CF6CAB31D9C539]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=46 numCloses=46
[junit4] 2> 668539 INFO (SUITE-ZkSolrClientTest-seed#[D1CF6CAB31D9C539]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields
[junit4] 2> 668557 INFO (SUITE-ZkSolrClientTest-seed#[D1CF6CAB31D9C539]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 668573 INFO (SUITE-ZkSolrClientTest-seed#[D1CF6CAB31D9C539]-worker) [ ] o.a.s.SolrTestCaseJ4 ####initCore
[junit4] 2> 668574 INFO (SUITE-ZkSolrClientTest-seed#[D1CF6CAB31D9C539]-worker) [ ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/lib, /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
[junit4] 2> 668793 WARN (SUITE-ZkSolrClientTest-seed#[D1CF6CAB31D9C539]-worker) [ ] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 668794 INFO (SUITE-ZkSolrClientTest-seed#[D1CF6CAB31D9C539]-worker) [ ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.3
[junit4] 2> 668886 INFO (SUITE-ZkSolrClientTest-seed#[D1CF6CAB31D9C539]-worker) [ ] o.a.s.s.IndexSchema [null] Schema name=test
[junit4] 2> 669437 INFO (SUITE-ZkSolrClientTest-seed#[D1CF6CAB31D9C539]-worker) [ ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 669508 INFO (SUITE-ZkSolrClientTest-seed#[D1CF6CAB31D9C539]-worker) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=30000&connTimeout=30000&retry=true
[junit4] 2> 669689 INFO (SUITE-ZkSolrClientTest-seed#[D1CF6CAB31D9C539]-worker) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@70a36c78
[junit4] 2> 669707 INFO (SUITE-ZkSolrClientTest-seed#[D1CF6CAB31D9C539]-worker) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@70a36c78
[junit4] 2> 669707 INFO (SUITE-ZkSolrClientTest-seed#[D1CF6CAB31D9C539]-worker) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@70a36c78
[junit4] 2> 669736 INFO (coreLoadExecutor-663-thread-1) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 669736 INFO (coreLoadExecutor-663-thread-1) [ x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/lib, /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
[junit4] 2> 669819 WARN (coreLoadExecutor-663-thread-1) [ x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 669841 INFO (coreLoadExecutor-663-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.3
[junit4] 2> 669916 INFO (coreLoadExecutor-663-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 670333 INFO (coreLoadExecutor-663-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 670356 INFO (coreLoadExecutor-663-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1, trusted=true
[junit4] 2> 670356 INFO (coreLoadExecutor-663-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@70a36c78
[junit4] 2> 670356 INFO (coreLoadExecutor-663-thread-1) [ x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 670356 INFO (coreLoadExecutor-663-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ZkSolrClientTest_D1CF6CAB31D9C539-001/init-core-data-001/]
[junit4] 2> 670357 INFO (coreLoadExecutor-663-thread-1) [ x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@70a36c78
[junit4] 2> 670371 INFO (coreLoadExecutor-663-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@e931a6a
[junit4] 2> 670642 INFO (coreLoadExecutor-663-thread-1) [ x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 670642 INFO (coreLoadExecutor-663-thread-1) [ x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 670643 INFO (coreLoadExecutor-663-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 670643 INFO (coreLoadExecutor-663-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 670644 INFO (coreLoadExecutor-663-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1713991106, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 670645 INFO (coreLoadExecutor-663-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@79dffdc0[collection1] main]
[junit4] 2> 670645 INFO (coreLoadExecutor-663-thread-1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
[junit4] 2> 670715 INFO (coreLoadExecutor-663-thread-1) [ x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4] 2> 670852 INFO (coreLoadExecutor-663-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 670852 INFO (searcherExecutor-664-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
[junit4] 2> 670853 INFO (searcherExecutor-664-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
[junit4] 2> 670853 INFO (searcherExecutor-664-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
[junit4] 2> 670853 INFO (searcherExecutor-664-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
[junit4] 2> 670853 INFO (searcherExecutor-664-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
[junit4] 2> 670854 INFO (searcherExecutor-664-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
[junit4] 2> 670854 INFO (searcherExecutor-664-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
[junit4] 2> 670854 INFO (searcherExecutor-664-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
[junit4] 2> 670854 INFO (searcherExecutor-664-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
[junit4] 2> 670856 INFO (searcherExecutor-664-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@79dffdc0[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 670874 INFO (coreLoadExecutor-663-thread-1) [ x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1594091586460844032
[junit4] 2> 670877 INFO (SUITE-ZkSolrClientTest-seed#[D1CF6CAB31D9C539]-worker) [ ] o.a.s.SolrTestCaseJ4 ####initCore end
[junit4] 2> 670897 INFO (TEST-ZkSolrClientTest.testReconnect-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testReconnect
[junit4] 2> 670897 INFO (TEST-ZkSolrClientTest.testReconnect-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 670898 INFO (Thread-3811) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 670910 INFO (Thread-3811) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 670956 ERROR (Thread-3811) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 671000 INFO (TEST-ZkSolrClientTest.testReconnect-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.ZkTestServer start zk server on port:42353
[junit4] 2> 671092 ERROR (TEST-ZkSolrClientTest.testReconnect-seed#[D1CF6CAB31D9C539]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 671093 INFO (TEST-ZkSolrClientTest.testReconnect-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:42353 42353
[junit4] 2> 671397 INFO (Thread-3811) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:42353 42353
[junit4] 2> 671497 INFO (TEST-ZkSolrClientTest.testReconnect-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 671497 WARN (zkCallback-933-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5ee3c059 name: ZooKeeperConnection Watcher:127.0.0.1:42353/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
[junit4] 2> 671497 WARN (zkCallback-933-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has disconnected
[junit4] 2> 671497 INFO (Thread-3812) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:42353
[junit4] 2> 671497 INFO (Thread-3812) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 671499 ERROR (Thread-3812) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 671597 INFO (TEST-ZkSolrClientTest.testReconnect-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.ZkTestServer start zk server on port:42353
[junit4] 2> 673058 WARN (zkCallback-933-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5ee3c059 name: ZooKeeperConnection Watcher:127.0.0.1:42353/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
[junit4] 2> 673059 WARN (zkCallback-933-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has disconnected
[junit4] 2> 674874 WARN (TEST-ZkSolrClientTest.testReconnect-seed#[D1CF6CAB31D9C539]-SendThread(127.0.0.1:42353)) [ ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x161f59da21c0002 has expired
[junit4] 2> 674882 WARN (zkCallback-933-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5ee3c059 name: ZooKeeperConnection Watcher:127.0.0.1:42353/solr got event WatchedEvent state:Expired type:None path:null path: null type: None
[junit4] 2> 674882 WARN (zkCallback-933-thread-1) [ ] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper...
[junit4] 2> 674882 WARN (zkCallback-933-thread-1) [ ] o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
[junit4] 2> 674935 INFO (zkCallback-933-thread-1) [ ] o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
[junit4] 2> 674935 INFO (zkCallback-933-thread-1) [ ] o.a.s.c.c.DefaultConnectionStrategy Reconnected to ZooKeeper
[junit4] 2> 674935 INFO (zkCallback-933-thread-1) [ ] o.a.s.c.c.ConnectionManager Connected:true
[junit4] 2> 675050 ERROR (TEST-ZkSolrClientTest.testReconnect-seed#[D1CF6CAB31D9C539]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 675054 INFO (TEST-ZkSolrClientTest.testReconnect-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:42353 42353
[junit4] 2> 675054 INFO (Thread-3812) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:42353 42353
[junit4] 2> 675055 INFO (TEST-ZkSolrClientTest.testReconnect-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testReconnect
[junit4] 2> 675089 INFO (TEST-ZkSolrClientTest.testConnect-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testConnect
[junit4] 2> 675091 INFO (TEST-ZkSolrClientTest.testConnect-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 675091 INFO (Thread-3813) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 675091 INFO (Thread-3813) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 675092 ERROR (Thread-3813) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 675191 INFO (TEST-ZkSolrClientTest.testConnect-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.ZkTestServer start zk server on port:40167
[junit4] 2> 675416 ERROR (TEST-ZkSolrClientTest.testConnect-seed#[D1CF6CAB31D9C539]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 675417 INFO (TEST-ZkSolrClientTest.testConnect-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:40167 40167
[junit4] 2> 675703 INFO (Thread-3813) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:40167 40167
[junit4] 2> 675714 INFO (TEST-ZkSolrClientTest.testConnect-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testConnect
[junit4] 2> 675717 INFO (TEST-ZkSolrClientTest.testMultipleWatchesAsync-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testMultipleWatchesAsync
[junit4] 2> 675717 INFO (TEST-ZkSolrClientTest.testMultipleWatchesAsync-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 675717 INFO (Thread-3814) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 675717 INFO (Thread-3814) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 675730 ERROR (Thread-3814) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 675821 INFO (TEST-ZkSolrClientTest.testMultipleWatchesAsync-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.ZkTestServer start zk server on port:43786
[junit4] 2> 679951 ERROR (TEST-ZkSolrClientTest.testMultipleWatchesAsync-seed#[D1CF6CAB31D9C539]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 679952 INFO (TEST-ZkSolrClientTest.testMultipleWatchesAsync-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:43786 43786
[junit4] 2> 680153 INFO (Thread-3814) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:43786 43786
[junit4] 2> 680154 INFO (TEST-ZkSolrClientTest.testMultipleWatchesAsync-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testMultipleWatchesAsync
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=ZkSolrClientTest -Dtests.method=testMultipleWatchesAsync -Dtests.seed=D1CF6CAB31D9C539 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/test-data/enwiki.random.lines.txt -Dtests.locale=hu-HU -Dtests.timezone=Etc/GMT-12 -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] FAILURE 4.46s J1 | ZkSolrClientTest.testMultipleWatchesAsync <<<
[junit4] > Throwable #1: java.lang.AssertionError
[junit4] > at __randomizedtesting.SeedInfo.seed([D1CF6CAB31D9C539:B979BF09A43DC4A7]:0)
[junit4] > at org.apache.solr.cloud.ZkSolrClientTest.testMultipleWatchesAsync(ZkSolrClientTest.java:257)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 680173 INFO (TEST-ZkSolrClientTest.testZkCmdExectutor-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testZkCmdExectutor
[junit4] 2> 680173 INFO (TEST-ZkSolrClientTest.testZkCmdExectutor-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 680174 INFO (Thread-3815) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 680174 INFO (Thread-3815) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 680175 ERROR (Thread-3815) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 680274 INFO (TEST-ZkSolrClientTest.testZkCmdExectutor-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.ZkTestServer start zk server on port:43309
[junit4] 2> 695293 ERROR (TEST-ZkSolrClientTest.testZkCmdExectutor-seed#[D1CF6CAB31D9C539]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 695293 INFO (TEST-ZkSolrClientTest.testZkCmdExectutor-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:43309 43309
[junit4] 2> 695395 INFO (Thread-3815) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:43309 43309
[junit4] 2> 695396 INFO (TEST-ZkSolrClientTest.testZkCmdExectutor-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testZkCmdExectutor
[junit4] 2> 695399 INFO (TEST-ZkSolrClientTest.testWatchChildren-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testWatchChildren
[junit4] 2> 695399 INFO (TEST-ZkSolrClientTest.testWatchChildren-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 695399 INFO (Thread-3816) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 695399 INFO (Thread-3816) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 695401 ERROR (Thread-3816) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 695499 INFO (TEST-ZkSolrClientTest.testWatchChildren-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.ZkTestServer start zk server on port:45455
[junit4] 2> 696243 ERROR (TEST-ZkSolrClientTest.testWatchChildren-seed#[D1CF6CAB31D9C539]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 696244 INFO (TEST-ZkSolrClientTest.testWatchChildren-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:45455 45455
[junit4] 2> 696308 INFO (Thread-3816) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:45455 45455
[junit4] 2> 696309 INFO (TEST-ZkSolrClientTest.testWatchChildren-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testWatchChildren
[junit4] 2> 696314 INFO (TEST-ZkSolrClientTest.testMakeRootNode-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testMakeRootNode
[junit4] 2> 696315 INFO (TEST-ZkSolrClientTest.testMakeRootNode-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 696315 INFO (Thread-3817) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 696315 INFO (Thread-3817) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 696328 ERROR (Thread-3817) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 696415 INFO (TEST-ZkSolrClientTest.testMakeRootNode-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.ZkTestServer start zk server on port:41630
[junit4] 2> 696681 ERROR (TEST-ZkSolrClientTest.testMakeRootNode-seed#[D1CF6CAB31D9C539]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 696714 INFO (TEST-ZkSolrClientTest.testMakeRootNode-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:41630 41630
[junit4] 2> 697043 INFO (Thread-3817) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:41630 41630
[junit4] 2> 697076 INFO (TEST-ZkSolrClientTest.testMakeRootNode-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testMakeRootNode
[junit4] 2> 697079 INFO (TEST-ZkSolrClientTest.testClean-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testClean
[junit4] 2> 697080 INFO (TEST-ZkSolrClientTest.testClean-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 697096 INFO (Thread-3818) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 697096 INFO (Thread-3818) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 697099 ERROR (Thread-3818) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 697196 INFO (TEST-ZkSolrClientTest.testClean-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.ZkTestServer start zk server on port:37246
[junit4] 2> 697415 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x161f59e08560002, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 697435 ERROR (TEST-ZkSolrClientTest.testClean-seed#[D1CF6CAB31D9C539]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 697435 INFO (TEST-ZkSolrClientTest.testClean-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:37246 37246
[junit4] 2> 697776 INFO (Thread-3818) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:37246 37246
[junit4] 2> 697777 INFO (TEST-ZkSolrClientTest.testClean-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testClean
[junit4] 2> 697792 INFO (TEST-ZkSolrClientTest.testSkipPathPartsOnMakePath-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testSkipPathPartsOnMakePath
[junit4] 2> 697792 INFO (TEST-ZkSolrClientTest.testSkipPathPartsOnMakePath-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 697794 INFO (Thread-3819) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 697794 INFO (Thread-3819) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 697800 ERROR (Thread-3819) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 697894 INFO (TEST-ZkSolrClientTest.testSkipPathPartsOnMakePath-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.ZkTestServer start zk server on port:41345
[junit4] 2> 697962 ERROR (TEST-ZkSolrClientTest.testSkipPathPartsOnMakePath-seed#[D1CF6CAB31D9C539]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 697963 INFO (TEST-ZkSolrClientTest.testSkipPathPartsOnMakePath-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:41345 41345
[junit4] 2> 698198 INFO (Thread-3819) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:41345 41345
[junit4] 2> 698199 INFO (TEST-ZkSolrClientTest.testSkipPathPartsOnMakePath-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testSkipPathPartsOnMakePath
[junit4] 2> 700201 INFO (SUITE-ZkSolrClientTest-seed#[D1CF6CAB31D9C539]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> 700201 INFO (SUITE-ZkSolrClientTest-seed#[D1CF6CAB31D9C539]-worker) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1723966582
[junit4] 2> 700201 INFO (SUITE-ZkSolrClientTest-seed#[D1CF6CAB31D9C539]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.node
[junit4] 2> 700201 INFO (SUITE-ZkSolrClientTest-seed#[D1CF6CAB31D9C539]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@648c1214: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@7d846b85
[junit4] 2> 700203 INFO (SUITE-ZkSolrClientTest-seed#[D1CF6CAB31D9C539]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.jvm
[junit4] 2> 700203 INFO (SUITE-ZkSolrClientTest-seed#[D1CF6CAB31D9C539]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5ccad9fb: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@2b283596
[junit4] 2> 700205 INFO (SUITE-ZkSolrClientTest-seed#[D1CF6CAB31D9C539]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.jetty
[junit4] 2> 700205 INFO (SUITE-ZkSolrClientTest-seed#[D1CF6CAB31D9C539]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5b581933: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@68a6bd3d
[junit4] 2> 700230 INFO (coreCloseExecutor-669-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@5bb70944
[junit4] 2> 700324 INFO (coreCloseExecutor-669-thread-1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.collection1
[junit4] 2> 700324 INFO (coreCloseExecutor-669-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@92a4ee0: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@578cc24e
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ZkSolrClientTest_D1CF6CAB31D9C539-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): {}, docValues:{}, maxPointsInLeafNode=991, maxMBSortInHeap=7.97808786154156, sim=RandomSimilarity(queryNorm=true,coord=crazy): {}, locale=hu-HU, timezone=Etc/GMT-12
[junit4] 2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=1,free=309858200,total=504365056
[junit4] 2> NOTE: All tests run in this JVM: [EchoParamsTest, TestHighFrequencyDictionaryFactory, TestCryptoKeys, DocValuesTest, MultiTermTest, BlobRepositoryCloudTest, TestUpdate, HdfsChaosMonkeySafeLeaderTest, TestNestedDocsSort, HighlighterTest, TestSolrCLIRunExample, HighlighterMaxOffsetTest, AddBlockUpdateTest, SolrSlf4jReporterTest, SuggestComponentTest, ZkSolrClientTest]
[junit4] Completed [58/714 (1!)] on J1 in 31.94s, 8 tests, 1 failure <<< FAILURES!
[...truncated 48 lines...]
[junit4] Suite: org.apache.solr.cloud.ShardSplitTest
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ShardSplitTest_D1CF6CAB31D9C539-001/init-core-data-001
[junit4] 2> 71494 WARN (SUITE-ShardSplitTest-seed#[D1CF6CAB31D9C539]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=4 numCloses=4
[junit4] 2> 71494 INFO (SUITE-ShardSplitTest-seed#[D1CF6CAB31D9C539]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields
[junit4] 2> 71496 INFO (SUITE-ShardSplitTest-seed#[D1CF6CAB31D9C539]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
[junit4] 2> 71496 INFO (SUITE-ShardSplitTest-seed#[D1CF6CAB31D9C539]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 2> 71639 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 71698 INFO (Thread-13) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 71700 INFO (Thread-13) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 72347 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.ZkTestServer start zk server on port:38692
[junit4] 2> 73476 ERROR (Thread-13) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 74912 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 75059 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 75064 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 75118 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 75155 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 75191 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 75265 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 75329 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 75410 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 75413 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 75548 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 77134 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ShardSplitTest_D1CF6CAB31D9C539-001/control-001/cores/collection1
[junit4] 2> 78501 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 78750 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@255ae242{/,null,AVAILABLE}
[junit4] 2> 78806 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@453c5d5f{HTTP/1.1,[http/1.1]}{127.0.0.1:34701}
[junit4] 2> 78806 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.e.j.s.Server Started @89774ms
[junit4] 2> 78807 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ShardSplitTest_D1CF6CAB31D9C539-001/tempDir-001/control/data, hostContext=/, hostPort=34701, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ShardSplitTest_D1CF6CAB31D9C539-001/control-001/cores}
[junit4] 2> 78896 ERROR (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 78896 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.6.3
[junit4] 2> 78896 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 78897 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 78930 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-03-05T09:53:31.699Z
[junit4] 2> 78931 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
[junit4] 2> 79115 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x161f5947e930002, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 79141 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 79156 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ShardSplitTest_D1CF6CAB31D9C539-001/control-001/solr.xml
[junit4] 2> 79403 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 79423 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38692/solr
[junit4] 2> 80494 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [n:127.0.0.1:34701_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 80529 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [n:127.0.0.1:34701_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:34701_
[junit4] 2> 80532 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [n:127.0.0.1:34701_ ] o.a.s.c.Overseer Overseer (id=99630684906389508-127.0.0.1:34701_-n_0000000000) starting
[junit4] 2> 81074 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [n:127.0.0.1:34701_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34701_
[junit4] 2> 81223 INFO (zkCallback-31-thread-1-processing-n:127.0.0.1:34701_) [n:127.0.0.1:34701_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 82521 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [n:127.0.0.1:34701_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ShardSplitTest_D1CF6CAB31D9C539-001/control-001/cores
[junit4] 2> 82522 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [n:127.0.0.1:34701_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 82698 INFO (coreLoadExecutor-154-thread-1-processing-n:127.0.0.1:34701_) [n:127.0.0.1:34701_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 83167 INFO (OverseerStateUpdate-99630684906389508-127.0.0.1:34701_-n_0000000000) [n:127.0.0.1:34701_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 84173 WARN (coreLoadExecutor-154-thread-1-processing-n:127.0.0.1:34701_) [n:127.0.0.1:34701_ c:control_collection x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 84173 INFO (coreLoadExecutor-154-thread-1-processing-n:127.0.0.1:34701_) [n:127.0.0.1:34701_ c:control_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.3
[junit4] 2> 84260 INFO (coreLoadExecutor-154-thread-1-processing-n:127.0.0.1:34701_) [n:127.0.0.1:34701_ c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 84605 INFO (coreLoadExecutor-154-thread-1-processing-n:127.0.0.1:34701_) [n:127.0.0.1:34701_ c:control_collection x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 84874 INFO (coreLoadExecutor-154-thread-1-processing-n:127.0.0.1:34701_) [n:127.0.0.1:34701_ c:control_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection, trusted=true
[junit4] 2> 84910 INFO (coreLoadExecutor-154-thread-1-processing-n:127.0.0.1:34701_) [n:127.0.0.1:34701_ c:control_collection x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 84910 INFO (coreLoadExecutor-154-thread-1-processing-n:127.0.0.1:34701_) [n:127.0.0.1:34701_ c:control_collection x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ShardSplitTest_D1CF6CAB31D9C539-001/control-001/cores/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ShardSplitTest_D1CF6CAB31D9C539-001/control-001/cores/collection1/data/]
[junit4] 2> 84946 INFO (coreLoadExecutor-154-thread-1-processing-n:127.0.0.1:34701_) [n:127.0.0.1:34701_ c:control_collection x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5105c032
[junit4] 2> 85022 INFO (coreLoadExecutor-154-thread-1-processing-n:127.0.0.1:34701_) [n:127.0.0.1:34701_ c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=588641108, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 85326 WARN (coreLoadExecutor-154-thread-1-processing-n:127.0.0.1:34701_) [n:127.0.0.1:34701_ c:control_collection x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 85982 INFO (coreLoadExecutor-154-thread-1-processing-n:127.0.0.1:34701_) [n:127.0.0.1:34701_ c:control_collection x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 85982 INFO (coreLoadExecutor-154-thread-1-processing-n:127.0.0.1:34701_) [n:127.0.0.1:34701_ c:control_collection x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 85983 INFO (coreLoadExecutor-154-thread-1-processing-n:127.0.0.1:34701_) [n:127.0.0.1:34701_ c:control_collection x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 85983 INFO (coreLoadExecutor-154-thread-1-processing-n:127.0.0.1:34701_) [n:127.0.0.1:34701_ c:control_collection x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 85984 INFO (coreLoadExecutor-154-thread-1-processing-n:127.0.0.1:34701_) [n:127.0.0.1:34701_ c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=33, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 86036 INFO (coreLoadExecutor-154-thread-1-processing-n:127.0.0.1:34701_) [n:127.0.0.1:34701_ c:control_collection x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@56b0aba[collection1] main]
[junit4] 2> 86116 INFO (coreLoadExecutor-154-thread-1-processing-n:127.0.0.1:34701_) [n:127.0.0.1:34701_ c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 86124 INFO (coreLoadExecutor-154-thread-1-processing-n:127.0.0.1:34701_) [n:127.0.0.1:34701_ c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 86124 INFO (coreLoadExecutor-154-thread-1-processing-n:127.0.0.1:34701_) [n:127.0.0.1:34701_ c:control_collection x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 86230 INFO (searcherExecutor-155-thread-1-processing-n:127.0.0.1:34701_ x:collection1 c:control_collection) [n:127.0.0.1:34701_ c:control_collection x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@56b0aba[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 86231 INFO (coreLoadExecutor-154-thread-1-processing-n:127.0.0.1:34701_) [n:127.0.0.1:34701_ c:control_collection x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1594090973070098432
[junit4] 2> 86380 INFO (coreZkRegister-147-thread-1-processing-n:127.0.0.1:34701_ x:collection1 c:control_collection) [n:127.0.0.1:34701_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 86380 INFO (coreZkRegister-147-thread-1-processing-n:127.0.0.1:34701_ x:collection1 c:control_collection) [n:127.0.0.1:34701_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 86393 INFO (coreZkRegister-147-thread-1-processing-n:127.0.0.1:34701_ x:collection1 c:control_collection) [n:127.0.0.1:34701_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:34701/collection1/
[junit4] 2> 86393 INFO (coreZkRegister-147-thread-1-processing-n:127.0.0.1:34701_ x:collection1 c:control_collection) [n:127.0.0.1:34701_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 86393 INFO (coreZkRegister-147-thread-1-processing-n:127.0.0.1:34701_ x:collection1 c:control_collection) [n:127.0.0.1:34701_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:34701/collection1/ has no replicas
[junit4] 2> 86393 INFO (coreZkRegister-147-thread-1-processing-n:127.0.0.1:34701_ x:collection1 c:control_collection) [n:127.0.0.1:34701_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 86672 INFO (coreZkRegister-147-thread-1-processing-n:127.0.0.1:34701_ x:collection1 c:control_collection) [n:127.0.0.1:34701_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:34701/collection1/ shard1
[junit4] 2> 86853 INFO (coreZkRegister-147-thread-1-processing-n:127.0.0.1:34701_ x:collection1 c:control_collection) [n:127.0.0.1:34701_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 87211 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 87226 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:38692/solr ready
[junit4] 2> 87227 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 87227 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
[junit4] 2> 88226 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ShardSplitTest_D1CF6CAB31D9C539-001/shard-1-001/cores/collection1
[junit4] 2> 88227 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ShardSplitTest_D1CF6CAB31D9C539-001/shard-1-001
[junit4] 2> 88262 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 88343 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4ddb013b{/,null,AVAILABLE}
[junit4] 2> 88344 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7a5f79ba{HTTP/1.1,[http/1.1]}{127.0.0.1:36188}
[junit4] 2> 88344 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.e.j.s.Server Started @99311ms
[junit4] 2> 88344 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ShardSplitTest_D1CF6CAB31D9C539-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=36188, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ShardSplitTest_D1CF6CAB31D9C539-001/shard-1-001/cores}
[junit4] 2> 88344 ERROR (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 88379 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.6.3
[junit4] 2> 88379 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 88379 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 88379 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-03-05T09:53:41.180Z
[junit4] 2> 88460 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 88460 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ShardSplitTest_D1CF6CAB31D9C539-001/shard-1-001/solr.xml
[junit4] 2> 88570 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 88586 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38692/solr
[junit4] 2> 88807 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [n:127.0.0.1:36188_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 88829 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [n:127.0.0.1:36188_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 88969 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [n:127.0.0.1:36188_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36188_
[junit4] 2> 88992 INFO (zkCallback-31-thread-1-processing-n:127.0.0.1:34701_) [n:127.0.0.1:34701_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 89041 INFO (zkCallback-41-thread-1-processing-n:127.0.0.1:36188_) [n:127.0.0.1:36188_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 89114 INFO (zkCallback-35-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 89434 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [n:127.0.0.1:36188_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ShardSplitTest_D1CF6CAB31D9C539-001/shard-1-001/cores
[junit4] 2> 89434 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[D1CF6CAB31D9C539]) [n:127.0.0.1:36188_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 89503 INFO (coreLoadExecutor-165-thread-1-processing-n:127.0.0.1:36188_) [n:127.0.0.1:36188_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 89602 INFO (OverseerStateUpdate-99630684906389508-127.0.0.1:34701_-n_0000000000) [n:127.0.0.1:34701_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 89785 INFO (zkCallback-41-thread-1-processing-n:127.0.0.1:36188_) [n:127.0.0.1:36188_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 90694 WARN (coreLoadExecutor-165-thread-1-processing-n:127.0.0.1:36188_) [n:127.0.0.1:36188_ c:collection1 x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 90747 INFO (coreLoadExecutor-165-thread-1-processing-n:127.0.0.1:36188_) [n:127.0.0.1:36188_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.3
[junit4] 2> 90857 INFO (coreLoadExecutor-165-thread-1-processing-n:127.0.0.1:36188_) [n:127.0.0.1:36188_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 91506 INFO (coreLoadExecutor-165-thread-1-processing-n:127.0.0.1:36188_) [n:127.0.0.1:36188_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 91788 INFO (coreLoadExecutor-165-thread-1-processing-n:127.0.0.1:36188_) [n:127.0.0.1:36188_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1, trusted=true
[junit4] 2> 91788 INFO (coreLoadExecutor-165-thread-1-processing-n:127.0.0.1:36188_) [n:127.0.0.1:36188_ c:collection1 x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 91788 INFO (coreLoadExecutor-165-thread-1-processing-n:127.0.0.1:36188_) [n:127.0.0.1:36188_ c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ShardSplitTest_D1CF6CAB31D9C539-001/shard-1-001/cores/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ShardSplitTest_D1CF6CAB31D9C539-001/shard-1-001/cores/collection1/data/]
[junit4] 2> 91789 INFO (coreLoadExecutor-165-thread-1-processing-n:127.0.0.1:36188_) [n:127.0.0.1:36188_ c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5105c032
[junit4] 2> 91803 INFO (coreLoadExecutor-165-thread-1-processing-n:127.0.0.1:36188_) [n:127.0.0.1:36188_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=588641108, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 91994 WARN (coreLoadExecutor-165-thread-1-processing-n:127.0.0.1:36188_) [n:127.0.0.1:36188_ c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 92913 INFO (coreLoadExecutor-165-thread-1-processing-n:127.0.0.1:36188_) [n:127.0.0.1:36188_ c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 92913 INFO (coreLoadExecutor-165-thread-1-processing-n:127.0.0.1:36188_) [n:127.0.0.1:36188_ c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 92914 INFO (coreLoadExecutor-165-thread-1-processing-n:127.0.0.1:36188_) [n:127.0.0.1:36188_ c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 92914 INFO (coreLoadExecutor-165-thread-1-processing-n:127.0.0.1:36188_) [n:127.0.0.1:36188_ c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 93040 INFO (coreLoadExecutor-165-thread-1-processing-n:127.0.0.1:36188_) [n:127.0.0.1:36188_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=33, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 93148 INFO (coreLoadExecutor-165-thread-1-processing-n:127.0.0.1:36188_) [n:127.0.0.1:36188_ c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@de20b51[collection1] main]
[junit4] 2> 93150 INFO (coreLoadExecutor-165-thread-1-processing-n:127.0.0.1:36188_) [n:127.0.0.1:36188_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 93150 INFO (coreLoadExecutor-165-thread-1-processing-n:127.0.0.1:36188_) [n:127.0.0.1:36188_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 93151 INFO (coreLoadExecutor-165-thread-1-processing-n:127.0.0.1:36188_) [n:127.0.0.1:36188_ c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 93313 INFO (searcherExecutor-166-thread-1-processing-n:127.0.0.1:36188_ x:collection1 c:collection1) [n:127.0.0.1:36188_ c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@de20b51[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 93314 INFO (coreLoadExecutor-165-thread-1-processing-n:127.0.0.1:36188_) [n:127.0.0.1:36188_ c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1594090980497162240
[junit4] 2> 93380 INFO (coreZkRegister-160-thread-1-processing-n:127.0.0.1:36188_ x:collection1 c:collection1) [n:127.0.0.1:36188_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to co
[...truncated too long message...]
... 51 more
[junit4] 2> 7744673 WARN (SUITE-StressHdfsTest-seed#[D1CF6CAB31D9C539]-worker) [ ] o.a.h.h.s.d.DirectoryScanner DirectoryScanner: shutdown has been called
[junit4] 2> 7744781 INFO (SUITE-StressHdfsTest-seed#[D1CF6CAB31D9C539]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
[junit4] 2> 7744819 WARN (1806785745@qtp-1893282705-1 - Acceptor0 HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41563) [ ] o.a.h.h.HttpServer2 HttpServer Acceptor: isRunning is false. Rechecking.
[junit4] 2> 7744819 WARN (1806785745@qtp-1893282705-1 - Acceptor0 HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41563) [ ] o.a.h.h.HttpServer2 HttpServer Acceptor: isRunning is false
[junit4] 2> 7744967 WARN (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.StressHdfsTest_D1CF6CAB31D9C539-001/tempDir-001/hdfsBaseDir/data/data3/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.StressHdfsTest_D1CF6CAB31D9C539-001/tempDir-001/hdfsBaseDir/data/data4/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:33234) [ ] o.a.h.h.s.d.IncrementalBlockReportManager IncrementalBlockReportManager interrupted
[junit4] 2> 7744967 WARN (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.StressHdfsTest_D1CF6CAB31D9C539-001/tempDir-001/hdfsBaseDir/data/data3/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.StressHdfsTest_D1CF6CAB31D9C539-001/tempDir-001/hdfsBaseDir/data/data4/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:33234) [ ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-668678065-127.0.0.1-1520250732384 (Datanode Uuid 15977575-7969-4b1b-a518-5c3b332a4218) service to lucene2-us-west.apache.org/127.0.0.1:33234
[junit4] 2> 7745042 WARN (SUITE-StressHdfsTest-seed#[D1CF6CAB31D9C539]-worker) [ ] o.a.h.h.s.d.DirectoryScanner DirectoryScanner: shutdown has been called
[junit4] 2> 7745374 INFO (SUITE-StressHdfsTest-seed#[D1CF6CAB31D9C539]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
[junit4] 2> 7745374 WARN (961297972@qtp-1371905720-1 - Acceptor0 HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33390) [ ] o.a.h.h.HttpServer2 HttpServer Acceptor: isRunning is false. Rechecking.
[junit4] 2> 7745441 WARN (961297972@qtp-1371905720-1 - Acceptor0 HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33390) [ ] o.a.h.h.HttpServer2 HttpServer Acceptor: isRunning is false
[junit4] 2> 7745705 WARN (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.StressHdfsTest_D1CF6CAB31D9C539-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.StressHdfsTest_D1CF6CAB31D9C539-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:33234) [ ] o.a.h.h.s.d.IncrementalBlockReportManager IncrementalBlockReportManager interrupted
[junit4] 2> 7745705 WARN (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.StressHdfsTest_D1CF6CAB31D9C539-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.StressHdfsTest_D1CF6CAB31D9C539-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:33234) [ ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-668678065-127.0.0.1-1520250732384 (Datanode Uuid dcf412c7-0bbc-427c-931b-294d1a7836eb) service to lucene2-us-west.apache.org/127.0.0.1:33234
[junit4] 2> 7746040 INFO (SUITE-StressHdfsTest-seed#[D1CF6CAB31D9C539]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@lucene2-us-west.apache.org:0
[junit4] 2> 7746040 WARN (800265047@qtp-235526813-1 - Acceptor0 HttpServer2$SelectChannelConnectorWithSafeStartup@lucene2-us-west.apache.org:46096) [ ] o.a.h.h.HttpServer2 HttpServer Acceptor: isRunning is false. Rechecking.
[junit4] 2> 7746040 WARN (800265047@qtp-235526813-1 - Acceptor0 HttpServer2$SelectChannelConnectorWithSafeStartup@lucene2-us-west.apache.org:46096) [ ] o.a.h.h.HttpServer2 HttpServer Acceptor: isRunning is false
[junit4] 2> 7746142 INFO (SUITE-StressHdfsTest-seed#[D1CF6CAB31D9C539]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.StressHdfsTest_D1CF6CAB31D9C539-001
[junit4] 2> Mar 05, 2018 12:01:19 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 34 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): {range_facet_l_dv=Lucene50(blocksize=128), multiDefault=Lucene50(blocksize=128), intDefault=PostingsFormat(name=MockRandom), id=Lucene50(blocksize=128), range_facet_i_dv=Lucene50(blocksize=128), text=PostingsFormat(name=LuceneVarGapFixedInterval), range_facet_l=Lucene50(blocksize=128), timestamp=Lucene50(blocksize=128), txt_t=PostingsFormat(name=LuceneVarGapFixedInterval)}, docValues:{range_facet_l_dv=DocValuesFormat(name=Memory), _version_=DocValuesFormat(name=Lucene54), range_facet_i_dv=DocValuesFormat(name=Lucene54), intDvoDefault=DocValuesFormat(name=Memory), timestamp=DocValuesFormat(name=Lucene54)}, maxPointsInLeafNode=812, maxMBSortInHeap=6.408973382775418, sim=RandomSimilarity(queryNorm=true,coord=crazy): {}, locale=zh-HK, timezone=Africa/Johannesburg
[junit4] 2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=1,free=115619736,total=534773760
[junit4] 2> NOTE: All tests run in this JVM: [EchoParamsTest, TestHighFrequencyDictionaryFactory, TestCryptoKeys, DocValuesTest, MultiTermTest, BlobRepositoryCloudTest, TestUpdate, HdfsChaosMonkeySafeLeaderTest, TestNestedDocsSort, HighlighterTest, TestSolrCLIRunExample, HighlighterMaxOffsetTest, AddBlockUpdateTest, SolrSlf4jReporterTest, SuggestComponentTest, ZkSolrClientTest, ConjunctionSolrSpellCheckerTest, SolrInfoMBeanTest, TestBlendedInfixSuggestions, TermsComponentTest, DeleteReplicaTest, TestLuceneMatchVersion, HLLUtilTest, InfoHandlerTest, TestQueryUtils, TestReloadAndDeleteDocs, TestStressVersions, ShufflingReplicaListTransformerTest, SystemInfoHandlerTest, TestPointFields, WrapperMergePolicyFactoryTest, TestFieldCacheVsDocValues, TestDistributedGrouping, TestExactStatsCache, DistribCursorPagingTest, SecurityConfHandlerTest, DistributedIntervalFacetingTest, SolrMetricReporterTest, TestSurroundQueryParser, TestSolrCoreParser, DistributedFacetPivotSmallAdvancedTest, BasicAuthStandaloneTest, TestDocBasedVersionConstraints, FileUtilsTest, DateRangeFieldTest, LeaderInitiatedRecoveryOnShardRestartTest, TermVectorComponentTest, SpellingQueryConverterTest, AssignTest, CdcrBootstrapTest, DateMathParserTest, OverseerStatusTest, TestDownShardTolerantSearch, TestComplexPhraseQParserPlugin, TestCoreAdminApis, SuggesterFSTTest, ShardRoutingTest, ExplicitHLLTest, TestCoreContainer, IgnoreCommitOptimizeUpdateProcessorFactoryTest, TestInPlaceUpdatesDistrib, TestFilteredDocIdSet, TestRecoveryHdfs, SolrGraphiteReporterTest, TestEmbeddedSolrServerConstructors, TestSolrQueryResponse, TestPayloadCheckQParserPlugin, TestNumericTerms32, HdfsRecoverLeaseTest, DistributedMLTComponentTest, JsonValidatorTest, TestManagedResourceStorage, DistributedFacetExistsSmallTest, BlockJoinFacetSimpleTest, DirectSolrSpellCheckerTest, TestRTGBase, SharedFSAutoReplicaFailoverTest, DeleteInactiveReplicaTest, MissingSegmentRecoveryTest, TestPivotHelperCode, TestDocTermOrds, CollectionStateFormat2Test, ZkCLITest, TestSolrCloudWithSecureImpersonation, LoggingHandlerTest, TestSQLHandler, TestQueryTypes, SoftAutoCommitTest, TestLMDirichletSimilarityFactory, SSLMigrationTest, TestFieldTypeCollectionResource, BigEndianAscendingWordDeserializerTest, TestJsonRequest, TestSolrConfigHandler, TestPartialUpdateDeduplication, MoreLikeThisHandlerTest, DocumentBuilderTest, OverseerCollectionConfigSetProcessorTest, TestApiFramework, TestReplicationHandlerBackup, TestCSVResponseWriter, TestReplicaProperties, SliceStateTest, TestLRUCache, SimplePostToolTest, CoreAdminCreateDiscoverTest, TolerantUpdateProcessorTest, TestBlobHandler, TestPerFieldSimilarityClassic, ConnectionReuseTest, TestMinMaxOnMultiValuedField, SolrCLIZkUtilsTest, TestLeaderElectionWithEmptyReplica, SchemaVersionSpecificBehaviorTest, ExitableDirectoryReaderTest, IndexSchemaTest, BasicDistributedZkTest, ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test, UnloadDistributedZkTest, CollectionsAPIDistributedZkTest, SyncSliceTest, OpenCloseCoreStressTest, OverseerTest, LeaderElectionIntegrationTest, TestReplicationHandler, TestStressReorder, TestRangeQuery, HttpPartitionTest, TestCloudRecovery, TestExclusionRuleCollectionAccess, TestHdfsCloudBackupRestore, TestLocalFSCloudBackupRestore, TestOnReconnectListenerSupport, TestRebalanceLeaders, TestSegmentSorting, TestStressLiveNodes, TestTolerantUpdateProcessorRandomCloud, TriLevelCompositeIdRoutingTest, HdfsWriteToMultipleCollectionsTest, StressHdfsTest]
[junit4] Completed [593/714 (5!)] on J1 in 555.92s, 1 test, 1 error <<< FAILURES!
[...truncated 46679 lines...]
[JENKINS] Lucene-Solr-NightlyTests-6.6 - Build # 34 - Failure
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.6/34/
5 tests failed.
FAILED: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testCollectionsAPI
Error Message:
Something is broken in the assert for no shards using the same indexDir - probably something was changed in the attributes published in the MBean of SolrCore : {}
Stack Trace:
java.lang.AssertionError: Something is broken in the assert for no shards using the same indexDir - probably something was changed in the attributes published in the MBean of SolrCore : {}
at __randomizedtesting.SeedInfo.seed([78CE862CA70D20DD:30BBF298A13E0F48]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.checkNoTwoShardsUseTheSameIndexDir(CollectionsAPIDistributedZkTest.java:646)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:524)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.handler.admin.MetricsHandlerTest.test
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([78CE862CA70D20DD:F09AB9F609F14D25]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at org.junit.Assert.assertNotNull(Assert.java:537)
at org.apache.solr.handler.admin.MetricsHandlerTest.test(MetricsHandlerTest.java:54)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.update.TestInPlaceUpdatesDistrib.test
Error Message:
Error from server at https://127.0.0.1:46742/t_r/y/collection1: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 500 </title> </head> <body> <h2>HTTP ERROR: 500</h2> <p>Problem accessing /t_r/y/collection1/update. Reason: <pre> java.lang.OutOfMemoryError: GC overhead limit exceeded</pre></p> <hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.14.v20161028</a><hr/> </body> </html>
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:46742/t_r/y/collection1: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 500 </title>
</head>
<body>
<h2>HTTP ERROR: 500</h2>
<p>Problem accessing /t_r/y/collection1/update. Reason:
<pre> java.lang.OutOfMemoryError: GC overhead limit exceeded</pre></p>
<hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.14.v20161028</a><hr/>
</body>
</html>
at __randomizedtesting.SeedInfo.seed([78CE862CA70D20DD:F09AB9F609F14D25]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:580)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:177)
at org.apache.solr.update.TestInPlaceUpdatesDistrib.addDocAndGetVersion(TestInPlaceUpdatesDistrib.java:1078)
at org.apache.solr.update.TestInPlaceUpdatesDistrib.buildRandomIndex(TestInPlaceUpdatesDistrib.java:1123)
at org.apache.solr.update.TestInPlaceUpdatesDistrib.docValuesUpdateTest(TestInPlaceUpdatesDistrib.java:327)
at org.apache.solr.update.TestInPlaceUpdatesDistrib.test(TestInPlaceUpdatesDistrib.java:154)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.update.TestInPlaceUpdatesDistrib
Error Message:
2 threads leaked from SUITE scope at org.apache.solr.update.TestInPlaceUpdatesDistrib: 1) Thread[id=63443, name=searcherExecutor-10694-thread-1, state=WAITING, group=TGRP-TestInPlaceUpdatesDistrib] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=63409, name=searcherExecutor-10683-thread-1, state=WAITING, group=TGRP-TestInPlaceUpdatesDistrib] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.update.TestInPlaceUpdatesDistrib:
1) Thread[id=63443, name=searcherExecutor-10694-thread-1, state=WAITING, group=TGRP-TestInPlaceUpdatesDistrib]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=63409, name=searcherExecutor-10683-thread-1, state=WAITING, group=TGRP-TestInPlaceUpdatesDistrib]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([78CE862CA70D20DD]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.update.TestInPlaceUpdatesDistrib
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=63443, name=searcherExecutor-10694-thread-1, state=WAITING, group=TGRP-TestInPlaceUpdatesDistrib] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=63409, name=searcherExecutor-10683-thread-1, state=WAITING, group=TGRP-TestInPlaceUpdatesDistrib] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=63443, name=searcherExecutor-10694-thread-1, state=WAITING, group=TGRP-TestInPlaceUpdatesDistrib]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=63409, name=searcherExecutor-10683-thread-1, state=WAITING, group=TGRP-TestInPlaceUpdatesDistrib]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([78CE862CA70D20DD]:0)
Build Log:
[...truncated 12004 lines...]
[junit4] Suite: org.apache.solr.handler.admin.MetricsHandlerTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.handler.admin.MetricsHandlerTest_78CE862CA70D20DD-001/init-core-data-001
[junit4] 2> 242026 WARN (SUITE-MetricsHandlerTest-seed#[78CE862CA70D20DD]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=6 numCloses=6
[junit4] 2> 242026 INFO (SUITE-MetricsHandlerTest-seed#[78CE862CA70D20DD]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields
[junit4] 2> 242028 INFO (SUITE-MetricsHandlerTest-seed#[78CE862CA70D20DD]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 242029 INFO (SUITE-MetricsHandlerTest-seed#[78CE862CA70D20DD]-worker) [ ] o.a.s.SolrTestCaseJ4 ####initCore
[junit4] 2> 242030 INFO (SUITE-MetricsHandlerTest-seed#[78CE862CA70D20DD]-worker) [ ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
[junit4] 2> 242081 WARN (SUITE-MetricsHandlerTest-seed#[78CE862CA70D20DD]-worker) [ ] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 242083 INFO (SUITE-MetricsHandlerTest-seed#[78CE862CA70D20DD]-worker) [ ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.3
[junit4] 2> 242114 INFO (SUITE-MetricsHandlerTest-seed#[78CE862CA70D20DD]-worker) [ ] o.a.s.s.IndexSchema [null] Schema name=test
[junit4] 2> 243050 INFO (SUITE-MetricsHandlerTest-seed#[78CE862CA70D20DD]-worker) [ ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 243062 INFO (SUITE-MetricsHandlerTest-seed#[78CE862CA70D20DD]-worker) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=30000&connTimeout=30000&retry=true
[junit4] 2> 243093 WARN (SUITE-MetricsHandlerTest-seed#[78CE862CA70D20DD]-worker) [ ] o.a.s.m.r.SolrJmxReporter No JMX server found. Not exposing Solr metrics via JMX.
[junit4] 2> 243093 WARN (SUITE-MetricsHandlerTest-seed#[78CE862CA70D20DD]-worker) [ ] o.a.s.m.r.SolrJmxReporter No JMX server found. Not exposing Solr metrics via JMX.
[junit4] 2> 243093 WARN (SUITE-MetricsHandlerTest-seed#[78CE862CA70D20DD]-worker) [ ] o.a.s.m.r.SolrJmxReporter No JMX server found. Not exposing Solr metrics via JMX.
[junit4] 2> 243094 INFO (coreLoadExecutor-1476-thread-1) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 243096 INFO (coreLoadExecutor-1476-thread-1) [ x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
[junit4] 2> 243133 WARN (coreLoadExecutor-1476-thread-1) [ x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 243135 INFO (coreLoadExecutor-1476-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.3
[junit4] 2> 243165 INFO (coreLoadExecutor-1476-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 243306 INFO (coreLoadExecutor-1476-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 243314 INFO (coreLoadExecutor-1476-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1, trusted=true
[junit4] 2> 243314 WARN (coreLoadExecutor-1476-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter No JMX server found. Not exposing Solr metrics via JMX.
[junit4] 2> 243314 INFO (coreLoadExecutor-1476-thread-1) [ x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 243315 INFO (coreLoadExecutor-1476-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.handler.admin.MetricsHandlerTest_78CE862CA70D20DD-001/init-core-data-001/]
[junit4] 2> 243317 INFO (coreLoadExecutor-1476-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=35, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.41176649114580666]
[junit4] 2> 243383 INFO (coreLoadExecutor-1476-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 243383 INFO (coreLoadExecutor-1476-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 243384 INFO (coreLoadExecutor-1476-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=6, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=1.0146484375, noCFSRatio=0.4752696713000235]
[junit4] 2> 243396 INFO (coreLoadExecutor-1476-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@75a7cdad[collection1] main]
[junit4] 2> 243397 INFO (coreLoadExecutor-1476-thread-1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
[junit4] 2> 243419 INFO (coreLoadExecutor-1476-thread-1) [ x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4] 2> 243456 INFO (coreLoadExecutor-1476-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 243456 INFO (searcherExecutor-1477-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
[junit4] 2> 243457 INFO (searcherExecutor-1477-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
[junit4] 2> 243457 INFO (searcherExecutor-1477-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
[junit4] 2> 243457 INFO (searcherExecutor-1477-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
[junit4] 2> 243457 INFO (searcherExecutor-1477-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
[junit4] 2> 243460 INFO (SUITE-MetricsHandlerTest-seed#[78CE862CA70D20DD]-worker) [ ] o.a.s.SolrTestCaseJ4 ####initCore end
[junit4] 2> 243461 INFO (searcherExecutor-1477-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
[junit4] 2> 243462 INFO (searcherExecutor-1477-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
[junit4] 2> 243463 INFO (searcherExecutor-1477-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
[junit4] 2> 243463 INFO (searcherExecutor-1477-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
[junit4] 2> 243464 INFO (searcherExecutor-1477-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@75a7cdad[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 243464 INFO (TEST-MetricsHandlerTest.testCompact-seed#[78CE862CA70D20DD]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testCompact
[junit4] 2> 243553 INFO (TEST-MetricsHandlerTest.testCompact-seed#[78CE862CA70D20DD]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testCompact
[junit4] 2> 243555 INFO (TEST-MetricsHandlerTest.test-seed#[78CE862CA70D20DD]) [ ] o.a.s.SolrTestCaseJ4 ###Starting test
[junit4] 2> 243581 INFO (TEST-MetricsHandlerTest.test-seed#[78CE862CA70D20DD]) [ ] o.a.s.SolrTestCaseJ4 ###Ending test
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=MetricsHandlerTest -Dtests.method=test -Dtests.seed=78CE862CA70D20DD -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/test-data/enwiki.random.lines.txt -Dtests.locale=es-VE -Dtests.timezone=Pacific/Kiritimati -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] FAILURE 0.03s J2 | MetricsHandlerTest.test <<<
[junit4] > Throwable #1: java.lang.AssertionError
[junit4] > at __randomizedtesting.SeedInfo.seed([78CE862CA70D20DD:F09AB9F609F14D25]:0)
[junit4] > at org.apache.solr.handler.admin.MetricsHandlerTest.test(MetricsHandlerTest.java:54)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 243584 INFO (SUITE-MetricsHandlerTest-seed#[78CE862CA70D20DD]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> 243584 INFO (SUITE-MetricsHandlerTest-seed#[78CE862CA70D20DD]-worker) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=178690372
[junit4] 2> 243584 INFO (SUITE-MetricsHandlerTest-seed#[78CE862CA70D20DD]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.node
[junit4] 2> 243584 INFO (SUITE-MetricsHandlerTest-seed#[78CE862CA70D20DD]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@25f00628: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / null
[junit4] 2> 243584 INFO (SUITE-MetricsHandlerTest-seed#[78CE862CA70D20DD]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.jvm
[junit4] 2> 243584 INFO (SUITE-MetricsHandlerTest-seed#[78CE862CA70D20DD]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2b4472db: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / null
[junit4] 2> 243584 INFO (SUITE-MetricsHandlerTest-seed#[78CE862CA70D20DD]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.jetty
[junit4] 2> 243584 INFO (SUITE-MetricsHandlerTest-seed#[78CE862CA70D20DD]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4ff1b81b: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / null
[junit4] 2> 243585 INFO (coreCloseExecutor-1481-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@1c8dd5cd
[junit4] 2> 243592 INFO (coreCloseExecutor-1481-thread-1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.collection1
[junit4] 2> 243592 INFO (coreCloseExecutor-1481-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@30a1bdef: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / null
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.handler.admin.MetricsHandlerTest_78CE862CA70D20DD-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): {}, docValues:{}, maxPointsInLeafNode=1765, maxMBSortInHeap=6.713281055499272, sim=RandomSimilarity(queryNorm=false,coord=no): {}, locale=es-VE, timezone=Pacific/Kiritimati
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=205503176,total=516947968
[junit4] 2> NOTE: All tests run in this JVM: [SolrJmxReporterCloudTest, RecoveryZkTest, TestCollationFieldDocValues, TestUnifiedSolrHighlighter, PluginInfoTest, TestLMJelinekMercerSimilarityFactory, DeleteLastCustomShardedReplicaTest, TestTestInjection, FieldMutatingUpdateProcessorTest, TestExceedMaxTermLength, TestAnalyzedSuggestions, TestRestManager, TestClusterProperties, TestMiniSolrCloudClusterSSL, TestPostingsSolrHighlighter, SolrCoreCheckLockOnStartupTest, OutOfBoxZkACLAndCredentialsProvidersTest, LeaderElectionTest, TestImpersonationWithHadoopAuth, TestSolrJ, TestSolrDynamicMBean, MoveReplicaTest, TestTrackingShardHandlerFactory, OverseerModifyCollectionTest, AddSchemaFieldsUpdateProcessorFactoryTest, TestCloudDeleteByQuery, TestRecovery, TestCopyFieldCollectionResource, QueryParsingTest, PrimUtilsTest, ClassificationUpdateProcessorFactoryTest, TestRawResponseWriter, SharedFSAutoReplicaFailoverUtilsTest, TestDynamicLoading, TestStressReorder, PeerSyncWithIndexFingerprintCachingTest, CloudExitableDirectoryReaderTest, HttpSolrCallGetCoreTest, MetricsHandlerTest]
[junit4] Completed [65/714 (1!)] on J2 in 1.59s, 2 tests, 1 failure <<< FAILURES!
[...truncated 1579 lines...]
[junit4] Suite: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_78CE862CA70D20DD-001/init-core-data-001
[junit4] 2> 3676714 WARN (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[78CE862CA70D20DD]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=48 numCloses=48
[junit4] 2> 3676715 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[78CE862CA70D20DD]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields
[junit4] 2> 3676716 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[78CE862CA70D20DD]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 3676717 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[78CE862CA70D20DD]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_78CE862CA70D20DD-001/tempDir-001
[junit4] 2> 3676717 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[78CE862CA70D20DD]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 3676717 INFO (Thread-38667) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 3676717 INFO (Thread-38667) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 3676724 ERROR (Thread-38667) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 3676817 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[78CE862CA70D20DD]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:51101
[junit4] 2> 3676824 INFO (jetty-launcher-4480-thread-1) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 3676829 INFO (jetty-launcher-4480-thread-3) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 3676829 INFO (jetty-launcher-4480-thread-2) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 3676829 INFO (jetty-launcher-4480-thread-4) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 3676837 INFO (jetty-launcher-4480-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@32e9e77b{/solr,null,AVAILABLE}
[junit4] 2> 3676843 INFO (jetty-launcher-4480-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1eac8a36{/solr,null,AVAILABLE}
[junit4] 2> 3676845 INFO (jetty-launcher-4480-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@164b0e94{/solr,null,AVAILABLE}
[junit4] 2> 3676849 INFO (jetty-launcher-4480-thread-4) [ ] o.e.j.s.AbstractConnector Started ServerConnector@13679d23{HTTP/1.1,[http/1.1]}{127.0.0.1:53244}
[junit4] 2> 3676849 INFO (jetty-launcher-4480-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7e8b00b0{HTTP/1.1,[http/1.1]}{127.0.0.1:35293}
[junit4] 2> 3676849 INFO (jetty-launcher-4480-thread-4) [ ] o.e.j.s.Server Started @3679779ms
[junit4] 2> 3676849 INFO (jetty-launcher-4480-thread-1) [ ] o.e.j.s.Server Started @3679779ms
[junit4] 2> 3676849 INFO (jetty-launcher-4480-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=53244}
[junit4] 2> 3676850 INFO (jetty-launcher-4480-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35293}
[junit4] 2> 3676850 ERROR (jetty-launcher-4480-thread-4) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3676850 INFO (jetty-launcher-4480-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.6.3
[junit4] 2> 3676850 INFO (jetty-launcher-4480-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3676850 INFO (jetty-launcher-4480-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3676850 INFO (jetty-launcher-4480-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-03-04T09:41:37Z
[junit4] 2> 3676850 ERROR (jetty-launcher-4480-thread-1) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3676851 INFO (jetty-launcher-4480-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.6.3
[junit4] 2> 3676851 INFO (jetty-launcher-4480-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3676851 INFO (jetty-launcher-4480-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3676851 INFO (jetty-launcher-4480-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-03-04T09:41:37.001Z
[junit4] 2> 3676850 INFO (jetty-launcher-4480-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@19072b2f{HTTP/1.1,[http/1.1]}{127.0.0.1:38931}
[junit4] 2> 3676852 INFO (jetty-launcher-4480-thread-2) [ ] o.e.j.s.Server Started @3679782ms
[junit4] 2> 3676852 INFO (jetty-launcher-4480-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=38931}
[junit4] 2> 3676852 ERROR (jetty-launcher-4480-thread-2) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3676852 INFO (jetty-launcher-4480-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.6.3
[junit4] 2> 3676852 INFO (jetty-launcher-4480-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3676852 INFO (jetty-launcher-4480-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3676852 INFO (jetty-launcher-4480-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-03-04T09:41:37.002Z
[junit4] 2> 3676853 INFO (jetty-launcher-4480-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@793c289{/solr,null,AVAILABLE}
[junit4] 2> 3676855 INFO (jetty-launcher-4480-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@65cc7b8c{HTTP/1.1,[http/1.1]}{127.0.0.1:57105}
[junit4] 2> 3676859 INFO (jetty-launcher-4480-thread-3) [ ] o.e.j.s.Server Started @3679789ms
[junit4] 2> 3676859 INFO (jetty-launcher-4480-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=57105}
[junit4] 2> 3676865 ERROR (jetty-launcher-4480-thread-3) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3676865 INFO (jetty-launcher-4480-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.6.3
[junit4] 2> 3676865 INFO (jetty-launcher-4480-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3676865 INFO (jetty-launcher-4480-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3676865 INFO (jetty-launcher-4480-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-03-04T09:41:37.015Z
[junit4] 2> 3676867 INFO (jetty-launcher-4480-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3676867 INFO (jetty-launcher-4480-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3676873 INFO (jetty-launcher-4480-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3676873 INFO (jetty-launcher-4480-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3676878 INFO (jetty-launcher-4480-thread-1) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 3676879 INFO (jetty-launcher-4480-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51101/solr
[junit4] 2> 3676882 INFO (jetty-launcher-4480-thread-2) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 3676883 INFO (jetty-launcher-4480-thread-4) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 3676884 INFO (jetty-launcher-4480-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51101/solr
[junit4] 2> 3676885 INFO (jetty-launcher-4480-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51101/solr
[junit4] 2> 3676886 INFO (jetty-launcher-4480-thread-3) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 3676886 INFO (jetty-launcher-4480-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51101/solr
[junit4] 2> 3676887 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x161f06353e50006, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 3676910 INFO (jetty-launcher-4480-thread-1) [n:127.0.0.1:35293_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3676910 INFO (jetty-launcher-4480-thread-4) [n:127.0.0.1:53244_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3676910 INFO (jetty-launcher-4480-thread-3) [n:127.0.0.1:57105_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3676910 INFO (jetty-launcher-4480-thread-2) [n:127.0.0.1:38931_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3676911 INFO (jetty-launcher-4480-thread-3) [n:127.0.0.1:57105_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:57105_solr
[junit4] 2> 3676912 INFO (jetty-launcher-4480-thread-4) [n:127.0.0.1:53244_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53244_solr
[junit4] 2> 3676912 INFO (jetty-launcher-4480-thread-2) [n:127.0.0.1:38931_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38931_solr
[junit4] 2> 3676912 INFO (jetty-launcher-4480-thread-1) [n:127.0.0.1:35293_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35293_solr
[junit4] 2> 3676912 INFO (jetty-launcher-4480-thread-3) [n:127.0.0.1:57105_solr ] o.a.s.c.Overseer Overseer (id=99624976178806796-127.0.0.1:57105_solr-n_0000000000) starting
[junit4] 2> 3676914 INFO (zkCallback-4500-thread-1-processing-n:127.0.0.1:53244_solr) [n:127.0.0.1:53244_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 3676914 INFO (zkCallback-4496-thread-1-processing-n:127.0.0.1:35293_solr) [n:127.0.0.1:35293_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 3676914 INFO (zkCallback-4502-thread-1-processing-n:127.0.0.1:57105_solr) [n:127.0.0.1:57105_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 3676922 INFO (zkCallback-4501-thread-1-processing-n:127.0.0.1:38931_solr) [n:127.0.0.1:38931_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 3676935 INFO (jetty-launcher-4480-thread-3) [n:127.0.0.1:57105_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57105_solr
[junit4] 2> 3676937 INFO (zkCallback-4501-thread-1-processing-n:127.0.0.1:38931_solr) [n:127.0.0.1:38931_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 3676937 INFO (zkCallback-4496-thread-1-processing-n:127.0.0.1:35293_solr) [n:127.0.0.1:35293_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 3676937 INFO (zkCallback-4500-thread-1-processing-n:127.0.0.1:53244_solr) [n:127.0.0.1:53244_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 3676937 INFO (zkCallback-4502-thread-1-processing-n:127.0.0.1:57105_solr) [n:127.0.0.1:57105_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 3676965 INFO (jetty-launcher-4480-thread-2) [n:127.0.0.1:38931_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_78CE862CA70D20DD-001/tempDir-001/node3/.
[junit4] 2> 3676965 INFO (jetty-launcher-4480-thread-4) [n:127.0.0.1:53244_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_78CE862CA70D20DD-001/tempDir-001/node4/.
[junit4] 2> 3676989 INFO (jetty-launcher-4480-thread-1) [n:127.0.0.1:35293_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_78CE862CA70D20DD-001/tempDir-001/node1/.
[junit4] 2> 3676995 INFO (jetty-launcher-4480-thread-3) [n:127.0.0.1:57105_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_78CE862CA70D20DD-001/tempDir-001/node2/.
[junit4] 2> 3677005 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[78CE862CA70D20DD]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 3677006 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[78CE862CA70D20DD]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:51101/solr ready
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 3677057 WARN (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[78CE862CA70D20DD]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 3677065 WARN (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[78CE862CA70D20DD]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 3677077 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[78CE862CA70D20DD]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 3677099 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[78CE862CA70D20DD]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_41817_hdfs____mmu3xn/webapp
[junit4] 2> 3677502 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[78CE862CA70D20DD]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41817
[junit4] 2> 3677560 WARN (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[78CE862CA70D20DD]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 3677561 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[78CE862CA70D20DD]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 3677575 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[78CE862CA70D20DD]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_45588_datanode____.a4fgw5/webapp
[junit4] 2> 3678003 INFO (SUITE-HdfsCollectionsAPIDistributedZkTest-seed#[78CE862CA70D20DD]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45588
[junit4] 2> 3678088 ERROR (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_78CE862CA70D20DD-001/tempDir-002/hdfsBaseDir/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_78CE862CA70D20DD-001/tempDir-002/hdfsBaseDir/data/data2/]] heartbeating to localhost/127.0.0.1:52843) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 3678094 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x48815576400c08: from storage DS-ff004887-7b11-462a-bc33-2e49b7d936dc node DatanodeRegistration(127.0.0.1:50209, datanodeUuid=61a0fe6e-62b9-4c1f-a6ae-3e29a3811d18, infoPort=51077, infoSecurePort=0, ipcPort=49138, storageInfo=lv=-56;cid=testClusterID;nsid=1545822465;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 3678094 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x48815576400c08: from storage DS-31e16253-85d5-470b-8675-76bdaa74acd0 node DatanodeRegistration(127.0.0.1:50209, datanodeUuid=61a0fe6e-62b9-4c1f-a6ae-3e29a3811d18, infoPort=51077, infoSecurePort=0, ipcPort=49138, storageInfo=lv=-56;cid=testClusterID;nsid=1545822465;c=0), blocks: 0, hasStaleStorage: false, processing time: 1 msecs
[junit4] 2> 3678208 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.addReplicaTest-seed#[78CE862CA70D20DD]) [ ] o.a.s.SolrTestCaseJ4 ###Starting addReplicaTest
[junit4] 2> 3678210 INFO (TEST-HdfsCollectionsAPIDistributedZkTest.addReplicaTest-seed#[78CE862CA70D20DD]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 3678214 INFO (qtp1613196478-56381) [n:127.0.0.1:53244_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf&maxShardsPerNode=4&name=addReplicaColl&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 3678217 INFO (OverseerThreadFactory-8695-thread-1-processing-n:127.0.0.1:57105_solr) [n:127.0.0.1:57105_solr ] o.a.s.c.CreateCollectionCmd Create collection addReplicaColl
[junit4] 2> 3678323 INFO (qtp990896924-56391) [n:127.0.0.1:38931_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=addReplicaColl_shard1_replica1&action=CREATE&numShards=2&collection=addReplicaColl&shard=shard1&wt=javabin&version=2
[junit4] 2> 3678323 INFO (qtp167393648-56371) [n:127.0.0.1:35293_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=addReplicaColl_shard2_replica1&action=CREATE&numShards=2&collection=addReplicaColl&shard=shard2&wt=javabin&version=2
[junit4] 2> 3678323 INFO (qtp990896924-56391) [n:127.0.0.1:38931_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 3678324 INFO (qtp167393648-56371) [n:127.0.0.1:35293_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 3678325 INFO (qtp1613196478-56379) [n:127.0.0.1:53244_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=addReplicaColl_shard1_replica2&action=CREATE&numShards=2&collection=addReplicaColl&shard=shard1&wt=javabin&version=2
[junit4] 2> 3678325 INFO (qtp1613196478-56379) [n:127.0.0.1:53244_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 3678326 INFO (qtp1299584871-56401) [n:127.0.0.1:57105_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=addReplicaColl_shard2_replica2&action=CREATE&numShards=2&collection=addReplicaColl&shard=shard2&wt=javabin&version=2
[junit4] 2> 3678326 INFO (qtp1299584871-56401) [n:127.0.0.1:57105_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 3678433 INFO (zkCallback-4500-thread-1-processing-n:127.0.0.1:53244_solr) [n:127.0.0.1:53244_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 3678433 INFO (zkCallback-4501-thread-1-processing-n:127.0.0.1:38931_solr) [n:127.0.0.1:38931_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 3678433 INFO (zkCallback-4496-thread-1-processing-n:127.0.0.1:35293_solr) [n:127.0.0.1:35293_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 3678433 INFO (zkCallback-4502-thread-1-processing-n:127.0.0.1:57105_solr) [n:127.0.0.1:57105_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 3679343 INFO (qtp1613196478-56379) [n:127.0.0.1:53244_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.3
[junit4] 2> 3679344 INFO (qtp167393648-56371) [n:127.0.0.1:35293_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.3
[junit4] 2> 3679344 INFO (qtp990896924-56391) [n:127.0.0.1:38931_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.3
[junit4] 2> 3679371 INFO (qtp1299584871-56401) [n:127.0.0.1:57105_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.3
[junit4] 2> 3679376 INFO (qtp990896924-56391) [n:127.0.0.1:38931_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.s.IndexSchema [addReplicaColl_shard1_replica1] Schema name=minimal
[junit4] 2> 3679379 INFO (qtp990896924-56391) [n:127.0.0.1:38931_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 3679379 INFO (qtp990896924-56391) [n:127.0.0.1:38931_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.c.CoreContainer Creating SolrCore 'addReplicaColl_shard1_replica1' using configuration from collection addReplicaColl, trusted=true
[junit4] 2> 3679380 INFO (qtp1613196478-56379) [n:127.0.0.1:53244_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.s.IndexSchema [addReplicaColl_shard1_replica2] Schema name=minimal
[junit4] 2> 3679380 INFO (qtp1299584871-56401) [n:127.0.0.1:57105_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.s.IndexSchema [addReplicaColl_shard2_replica2] Schema name=minimal
[junit4] 2> 3679382 INFO (qtp1299584871-56401) [n:127.0.0.1:57105_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 3679382 INFO (qtp1299584871-56401) [n:127.0.0.1:57105_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.c.CoreContainer Creating SolrCore 'addReplicaColl_shard2_replica2' using configuration from collection addReplicaColl, trusted=true
[junit4] 2> 3679384 INFO (qtp1613196478-56379) [n:127.0.0.1:53244_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 3679384 INFO (qtp1613196478-56379) [n:127.0.0.1:53244_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.c.CoreContainer Creating SolrCore 'addReplicaColl_shard1_replica2' using configuration from collection addReplicaColl, trusted=true
[junit4] 2> 3679388 INFO (qtp990896924-56391) [n:127.0.0.1:38931_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:52843/data
[junit4] 2> 3679388 INFO (qtp990896924-56391) [n:127.0.0.1:38931_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 3679388 INFO (qtp990896924-56391) [n:127.0.0.1:38931_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 3679388 INFO (qtp1613196478-56379) [n:127.0.0.1:53244_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:52843/data
[junit4] 2> 3679388 INFO (qtp990896924-56391) [n:127.0.0.1:38931_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.c.SolrCore [[addReplicaColl_shard1_replica1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_78CE862CA70D20DD-001/tempDir-001/node3/addReplicaColl_shard1_replica1], dataDir=[hdfs://localhost:52843/data/addReplicaColl/core_node1/data/]
[junit4] 2> 3679388 INFO (qtp1613196478-56379) [n:127.0.0.1:53244_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 3679388 INFO (qtp1613196478-56379) [n:127.0.0.1:53244_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 3679388 INFO (qtp1613196478-56379) [n:127.0.0.1:53244_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.c.SolrCore [[addReplicaColl_shard1_replica2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_78CE862CA70D20DD-001/tempDir-001/node4/addReplicaColl_shard1_replica2], dataDir=[hdfs://localhost:52843/data/addReplicaColl/core_node3/data/]
[junit4] 2> 3679389 INFO (qtp1299584871-56401) [n:127.0.0.1:57105_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:52843/data
[junit4] 2> 3679389 INFO (qtp1299584871-56401) [n:127.0.0.1:57105_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 3679389 INFO (qtp1299584871-56401) [n:127.0.0.1:57105_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 3679389 INFO (qtp1299584871-56401) [n:127.0.0.1:57105_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.c.SolrCore [[addReplicaColl_shard2_replica2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_78CE862CA70D20DD-001/tempDir-001/node2/addReplicaColl_shard2_replica2], dataDir=[hdfs://localhost:52843/data/addReplicaColl/core_node4/data/]
[junit4] 2> 3679389 INFO (qtp990896924-56391) [n:127.0.0.1:38931_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:52843/data/addReplicaColl/core_node1/data/snapshot_metadata
[junit4] 2> 3679389 INFO (qtp1613196478-56379) [n:127.0.0.1:53244_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:52843/data/addReplicaColl/core_node3/data/snapshot_metadata
[junit4] 2> 3679389 INFO (qtp1299584871-56401) [n:127.0.0.1:57105_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:52843/data/addReplicaColl/core_node4/data/snapshot_metadata
[junit4] 2> 3679393 INFO (qtp167393648-56371) [n:127.0.0.1:35293_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica1] o.a.s.s.IndexSchema [addReplicaColl_shard2_replica1] Schema name=minimal
[junit4] 2> 3679396 INFO (qtp167393648-56371) [n:127.0.0.1:35293_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 3679396 INFO (qtp167393648-56371) [n:127.0.0.1:35293_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica1] o.a.s.c.CoreContainer Creating SolrCore 'addReplicaColl_shard2_replica1' using configuration from collection addReplicaColl, trusted=true
[junit4] 2> 3679396 INFO (qtp167393648-56371) [n:127.0.0.1:35293_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:52843/data
[junit4] 2> 3679396 INFO (qtp167393648-56371) [n:127.0.0.1:35293_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 3679397 INFO (qtp167393648-56371) [n:127.0.0.1:35293_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 3679397 INFO (qtp167393648-56371) [n:127.0.0.1:35293_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica1] o.a.s.c.SolrCore [[addReplicaColl_shard2_replica1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest_78CE862CA70D20DD-001/tempDir-001/node1/addReplicaColl_shard2_replica1], dataDir=[hdfs://localhost:52843/data/addReplicaColl/core_node2/data/]
[junit4] 2> 3679397 INFO (qtp167393648-56371) [n:127.0.0.1:35293_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:52843/data/addReplicaColl/core_node2/data/snapshot_metadata
[junit4] 2> 3679413 INFO (qtp1613196478-56379) [n:127.0.0.1:53244_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 3679413 INFO (qtp1613196478-56379) [n:127.0.0.1:53244_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
[junit4] 2> 3679413 INFO (qtp1613196478-56379) [n:127.0.0.1:53244_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 3679415 INFO (qtp990896924-56391) [n:127.0.0.1:38931_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 3679415 INFO (qtp990896924-56391) [n:127.0.0.1:38931_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
[junit4] 2> 3679415 INFO (qtp990896924-56391) [n:127.0.0.1:38931_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 3679415 INFO (qtp1299584871-56401) [n:127.0.0.1:57105_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 3679415 INFO (qtp167393648-56371) [n:127.0.0.1:35293_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 3679415 INFO (qtp1299584871-56401) [n:127.0.0.1:57105_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
[junit4] 2> 3679416 INFO (qtp167393648-56371) [n:127.0.0.1:35293_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
[junit4] 2> 3679416 INFO (qtp1299584871-56401) [n:127.0.0.1:57105_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 3679416 INFO (qtp167393648-56371) [n:127.0.0.1:35293_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 3679424 WARN (qtp167393648-56371) [n:127.0.0.1:35293_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica1] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
[junit4] 2> 3679424 WARN (qtp1613196478-56379) [n:127.0.0.1:53244_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
[junit4] 2> 3679425 WARN (qtp1299584871-56401) [n:127.0.0.1:57105_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
[junit4] 2> 3679426 WARN (qtp990896924-56391) [n:127.0.0.1:38931_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
[junit4] 2> 3684431 INFO (qtp1613196478-56379) [n:127.0.0.1:53244_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 3684433 INFO (qtp167393648-56371) [n:127.0.0.1:35293_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 3684436 INFO (qtp1299584871-56401) [n:127.0.0.1:57105_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 3684436 INFO (qtp1613196478-56379) [n:127.0.0.1:53244_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:52843/data/addReplicaColl/core_node3/data
[junit4] 2> 3684443 INFO (qtp990896924-56391) [n:127.0.0.1:38931_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 3684443 INFO (qtp1299584871-56401) [n:127.0.0.1:57105_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:52843/data/addReplicaColl/core_node4/data
[junit4] 2> 3684446 INFO (qtp167393648-56371) [n:127.0.0.1:35293_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:52843/data/addReplicaColl/core_node2/data
[junit4] 2> 3684447 INFO (qtp990896924-56391) [n:127.0.0.1:38931_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:52843/data/addReplicaColl/core_node1/data
[junit4] 2> 3684468 INFO (qtp1613196478-56379) [n:127.0.0.1:53244_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:52843/data/addReplicaColl/core_node3/data/index
[junit4] 2> 3684476 INFO (qtp1613196478-56379) [n:127.0.0.1:53244_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 3684477 INFO (qtp1613196478-56379) [n:127.0.0.1:53244_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
[junit4] 2> 3684477 INFO (qtp1613196478-56379) [n:127.0.0.1:53244_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 3684477 INFO (qtp1299584871-56401) [n:127.0.0.1:57105_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:52843/data/addReplicaColl/core_node4/data/index
[junit4] 2> 3684481 INFO (qtp1613196478-56379) [n:127.0.0.1:53244_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 3684485 INFO (qtp990896924-56391) [n:127.0.0.1:38931_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:52843/data/addReplicaColl/core_node1/data/index
[junit4] 2> 3684486 INFO (qtp1299584871-56401) [n:127.0.0.1:57105_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 3684486 INFO (qtp1299584871-56401) [n:127.0.0.1:57105_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
[junit4] 2> 3684486 INFO (qtp1299584871-56401) [n:127.0.0.1:57105_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 3684493 INFO (qtp1299584871-56401) [n:127.0.0.1:57105_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 3684495 INFO (qtp167393648-56371) [n:127.0.0.1:35293_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:52843/data/addReplicaColl/core_node2/data/index
[junit4] 2> 3684497 INFO (qtp990896924-56391) [n:127.0.0.1:38931_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 3684497 INFO (qtp990896924-56391) [n:127.0.0.1:38931_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
[junit4] 2> 3684497 INFO (qtp990896924-56391) [n:127.0.0.1:38931_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 3684503 INFO (qtp990896924-56391) [n:127.0.0.1:38931_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 3684509 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:50209 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-31e16253-85d5-470b-8675-76bdaa74acd0:NORMAL:127.0.0.1:50209|RBW]]} size 0
[junit4] 2> 3684516 INFO (qtp167393648-56371) [n:127.0.0.1:35293_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 3684516 INFO (qtp167393648-56371) [n:127.0.0.1:35293_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
[junit4] 2> 3684516 INFO (qtp167393648-56371) [n:127.0.0.1:35293_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 3684523 INFO (qtp167393648-56371) [n:127.0.0.1:35293_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 3684536 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:50209 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-ff004887-7b11-462a-bc33-2e49b7d936dc:NORMAL:127.0.0.1:50209|FINALIZED]]} size 0
[junit4] 2> 3684537 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:50209 is added to blk_1073741827_1003{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-31e16253-85d5-470b-8675-76bdaa74acd0:NORMAL:127.0.0.1:50209|RBW]]} size 0
[junit4] 2> 3684598 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:50209 is added to blk_1073741828_1004{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-ff004887-7b11-462a-bc33-2e49b7d936dc:NORMAL:127.0.0.1:50209|FINALIZED]]} size 0
[junit4] 2> 3684599 INFO (qtp1613196478-56379) [n:127.0.0.1:53244_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 3684599 INFO (qtp1613196478-56379) [n:127.0.0.1:53244_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 3684599 INFO (qtp1613196478-56379) [n:127.0.0.1:53244_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
[junit4] 2> 3684626 INFO (qtp1613196478-56379) [n:127.0.0.1:53244_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 3684626 INFO (qtp1613196478-56379) [n:127.0.0.1:53244_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 3684635 INFO (qtp990896924-56391) [n:127.0.0.1:38931_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 3684635 INFO (qtp990896924-56391) [n:127.0.0.1:38931_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 3684635 INFO (qtp990896924-56391) [n:127.0.0.1:38931_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
[junit4] 2> 3684638 INFO (qtp1299584871-56401) [n:127.0.0.1:57105_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 3684638 INFO (qtp1299584871-56401) [n:127.0.0.1:57105_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 3684638 INFO (qtp1299584871-56401) [n:127.0.0.1:57105_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
[junit4] 2> 3684652 INFO (qtp990896924-56391) [n:127.0.0.1:38931_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 3684652 INFO (qtp1613196478-56379) [n:127.0.0.1:53244_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.s.SolrIndexSearcher Opening [Searcher@107e8137[addReplicaColl_shard1_replica2] main]
[junit4] 2> 3684652 INFO (qtp990896924-56391) [n:127.0.0.1:38931_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 3684654 INFO (qtp1613196478-56379) [n:127.0.0.1:53244_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 3684654 INFO (qtp1613196478-56379) [n:127.0.0.1:53244_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 3684654 INFO (qtp1613196478-56379) [n:127.0.0.1:53244_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 3684655 INFO (qtp1613196478-56379) [n:127.0.0.1:53244_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1593999627182407680
[junit4] 2> 3684657 INFO (searcherExecutor-8706-thread-1-processing-n:127.0.0.1:53244_solr x:addReplicaColl_shard1_replica2 s:shard1 c:addReplicaColl) [n:127.0.0.1:53244_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.c.SolrCore [addReplicaColl_shard1_replica2] Registered new searcher Searcher@107e8137[addReplicaColl_shard1_replica2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 3684661 INFO (qtp1299584871-56401) [n:127.0.0.1:57105_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 3684661 INFO (qtp1299584871-56401) [n:127.0.0.1:57105_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 3684663 INFO (qtp1613196478-56379) [n:127.0.0.1:53244_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9999ms
[junit4] 2> 3684665 INFO (zkCallback-4501-thread-1-processing-n:127.0.0.1:38931_solr) [n:127.0.0.1:38931_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 3684665 INFO (zkCallback-4496-thread-1-processing-n:127.0.0.1:35293_solr) [n:127.0.0.1:35293_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 3684665 INFO (zkCallback-4502-thread-1-processing-n:127.0.0.1:57105_solr) [n:127.0.0.1:57105_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 3684665 INFO (zkCallback-4500-thread-1-processing-n:127.0.0.1:53244_solr) [n:127.0.0.1:53244_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 3684672 INFO (qtp167393648-56371) [n:127.0.0.1:35293_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 3684672 INFO (qtp167393648-56371) [n:127.0.0.1:35293_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 3684672 INFO (qtp167393648-56371) [n:127.0.0.1:35293_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
[junit4] 2> 3684678 INFO (qtp990896924-56391) [n:127.0.0.1:38931_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@446c4372[addReplicaColl_shard1_replica1] main]
[junit4] 2> 3684680 INFO (qtp990896924-56391) [n:127.0.0.1:38931_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 3684681 INFO (qtp990896924-56391) [n:127.0.0.1:38931_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 3684681 INFO (qtp1299584871-56401) [n:127.0.0.1:57105_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.s.SolrIndexSearcher Opening [Searcher@655b13ba[addReplicaColl_shard2_replica2] main]
[junit4] 2> 3684681 INFO (qtp990896924-56391) [n:127.0.0.1:38931_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 3684681 INFO (qtp990896924-56391) [n:127.0.0.1:38931_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1593999627209670656
[junit4] 2> 3684682 INFO (searcherExecutor-8704-thread-1-processing-n:127.0.0.1:38931_solr x:addReplicaColl_shard1_replica1 s:shard1 c:addReplicaColl) [n:127.0.0.1:38931_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.c.SolrCore [addReplicaColl_shard1_replica1] Registered new searcher Searcher@446c4372[addReplicaColl_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 3684683 INFO (qtp1299584871-56401) [n:127.0.0.1:57105_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 3684683 INFO (qtp1299584871-56401) [n:127.0.0.1:57105_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 3684683 INFO (qtp1299584871-56401) [n:127.0.0.1:57105_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.h.Replicati
[...truncated too long message...]
entDeleteAndCreateCollectionTest, OverseerTaskQueueTest, TestSort, CloudMLTQParserTest, CoreSorterTest, TestShardHandlerFactory, TestConfigSets, AsyncCallRequestStatusResponseTest, CoreAdminRequestStatusTest, RuleEngineTest, PKIAuthenticationIntegrationTest, TestTrieFacet, TestSolrQueryParserDefaultOperatorResource, LeaderFailureAfterFreshStartTest, TestPerFieldSimilarityWithDefaultOverride, TestFieldCacheSort, OpenExchangeRatesOrgProviderTest, TestObjectReleaseTracker, ShowFileRequestHandlerTest, HdfsBasicDistributedZkTest, PreAnalyzedFieldManagedSchemaCloudTest, TestSegmentSorting, PreAnalyzedFieldTest, ParsingFieldUpdateProcessorsTest, NoCacheHeaderTest, TestChildDocTransformer, BlockJoinFacetDistribTest, SolrMetricsIntegrationTest, TestMergePolicyConfig, StatelessScriptUpdateProcessorFactoryTest, ZkStateReaderTest, TestNumericTerms64, TestConfigOverlay, CursorMarkTest, TestFieldCollectionResource, LegacyCloudClusterPropTest, SolrCoreMetricManagerTest, DistributedVersionInfoTest, TestJsonFacetRefinement, DebugComponentTest, UnloadDistributedZkTest, OverseerTest, LeaderElectionIntegrationTest, BasicZkTest, FullSolrCloudDistribCmdsTest, TestHashPartitioner, TermVectorComponentDistributedTest, TestReload, TestRangeQuery, HighlighterConfigTest, TestQuerySenderListener, ResponseLogComponentTest, SolrIndexConfigTest, BadComponentTest, TestStressRecovery, SynonymTokenizerTest, TestXIncludeConfig, EchoParamsTest, TestIBSimilarityFactory, TestFastWriter, TimeZoneUtilsTest, ScriptEngineTest, ChaosMonkeyNothingIsSafeTest, CdcrReplicationDistributedZkTest, CdcrReplicationHandlerTest, ConfigSetsAPITest, CreateCollectionCleanupTest, DistribCursorPagingTest, ForceLeaderTest, SSLMigrationTest, SaslZkACLProviderTest, ShardSplitTest, TestCloudPivotFacet, TestCloudPseudoReturnFields, TestCloudRecovery, TestCollectionAPI, TestCryptoKeys, TestLeaderElectionWithEmptyReplica, TestLeaderElectionZkExpiry, TestLockTree, TestMiniSolrCloudCluster, TestRandomRequestDistribution, TestRebalanceLeaders, TestReplicaProperties, TestRequestForwarding, TestSSLRandomization, TestSizeLimitedDistributedMap, TestTolerantUpdateProcessorCloud, TestTolerantUpdateProcessorRandomCloud, TriLevelCompositeIdRoutingTest, VMParamsZkACLAndCredentialsProvidersTest, HdfsBasicDistributedZk2Test, HdfsCollectionsAPIDistributedZkTest, TestClusterStateMutator, RulesTest, HdfsDirectoryFactoryTest, TestConfigSetImmutable, TestConfigSetProperties, TestCustomStream, TestImplicitCoreProperties, TestNRTOpen, TestSolrConfigHandler, TestHdfsBackupRestoreCore, CoreAdminCreateDiscoverTest, CoreMergeIndexesAdminHandlerTest, InfoHandlerTest, SecurityConfHandlerTest, StatsReloadRaceTest, TestCollectionAPIs, TestCoreAdminApis, DistributedExpandComponentTest, DistributedFacetPivotSmallTest, DistributedSuggestComponentTest, JavabinLoaderTest, HighlighterMaxOffsetTest, TestSlowCompositeReaderWrapper, WrapperMergePolicyFactoryTest, CheckHdfsIndexTest, TestCloudSchemaless, TestUseDocValuesAsStored2, RankQueryTest, SortSpecParsingTest, TestCollapseQParserPlugin, TestComplexPhraseLeadingWildcard, TestFieldSortValues, TestFilteredDocIdSet, TestHashQParserPlugin, TestNoOpRegenerator, TestRandomCollapseQParserPlugin, TestSmileRequest, TestStandardQParsers, TestXmlQParser, TestXmlQParserPlugin, TestJsonFacets, TestAuthorizationFramework, TestFieldCacheReopen, TestFieldCacheSortRandom, TestLegacyFieldCache, TestNumericTerms32, TestUninvertingReader, AddBlockUpdateTest, AnalysisErrorHandlingTest, CdcrUpdateLogTest, TestInPlaceUpdatesDistrib]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestInPlaceUpdatesDistrib -Dtests.seed=78CE862CA70D20DD -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/test-data/enwiki.random.lines.txt -Dtests.locale=es-CU -Dtests.timezone=Etc/Zulu -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J2 | TestInPlaceUpdatesDistrib (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.update.TestInPlaceUpdatesDistrib:
[junit4] > 1) Thread[id=63443, name=searcherExecutor-10694-thread-1, state=WAITING, group=TGRP-TestInPlaceUpdatesDistrib]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 2) Thread[id=63409, name=searcherExecutor-10683-thread-1, state=WAITING, group=TGRP-TestInPlaceUpdatesDistrib]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([78CE862CA70D20DD]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=63443, name=searcherExecutor-10694-thread-1, state=WAITING, group=TGRP-TestInPlaceUpdatesDistrib]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 2) Thread[id=63409, name=searcherExecutor-10683-thread-1, state=WAITING, group=TGRP-TestInPlaceUpdatesDistrib]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([78CE862CA70D20DD]:0)
[junit4] Completed [714/714 (3!)] on J2 in 1465.25s, 1 test, 3 errors <<< FAILURES!
[...truncated 1 lines...]
[junit4] JVM J2: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/temp/junit4-J2-20180304_084016_6928344625705321710958.sysout
[junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
[junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
[junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/heapdumps/java_pid7833.hprof ...
[junit4] Heap dump file created [660867319 bytes in 3.026 secs]
[junit4] <<< JVM J2: EOF ----
[junit4] JVM J2: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/temp/junit4-J2-20180304_084016_692805811283229828314.syserr
[junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
[junit4]
[junit4] Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "SessionTracker"
[junit4] <<< JVM J2: EOF ----
[...truncated 8104 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/build.xml:783: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/build.xml:727: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid7833.hprof
Total time: 277 minutes 55 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-NightlyTests-6.6 - Build # 33 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.6/33/
2 tests failed.
FAILED: org.apache.solr.cloud.MissingSegmentRecoveryTest.testLeaderRecovery
Error Message:
Expected a collection with one shard and two replicas null Last available state: DocCollection(MissingSegmentRecoveryTest//collections/MissingSegmentRecoveryTest/state.json/7)={ "replicationFactor":"2", "shards":{"shard1":{ "range":"80000000-7fffffff", "state":"active", "replicas":{ "core_node1":{ "core":"MissingSegmentRecoveryTest_shard1_replica1", "base_url":"https://127.0.0.1:42561/solr", "node_name":"127.0.0.1:42561_solr", "state":"active", "leader":"true"}, "core_node2":{ "core":"MissingSegmentRecoveryTest_shard1_replica2", "base_url":"https://127.0.0.1:35061/solr", "node_name":"127.0.0.1:35061_solr", "state":"down"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"false"}
Stack Trace:
java.lang.AssertionError: Expected a collection with one shard and two replicas
null
Last available state: DocCollection(MissingSegmentRecoveryTest//collections/MissingSegmentRecoveryTest/state.json/7)={
"replicationFactor":"2",
"shards":{"shard1":{
"range":"80000000-7fffffff",
"state":"active",
"replicas":{
"core_node1":{
"core":"MissingSegmentRecoveryTest_shard1_replica1",
"base_url":"https://127.0.0.1:42561/solr",
"node_name":"127.0.0.1:42561_solr",
"state":"active",
"leader":"true"},
"core_node2":{
"core":"MissingSegmentRecoveryTest_shard1_replica2",
"base_url":"https://127.0.0.1:35061/solr",
"node_name":"127.0.0.1:35061_solr",
"state":"down"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false"}
at __randomizedtesting.SeedInfo.seed([9B90977827D05434:CBC50F7B7EF1E229]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:265)
at org.apache.solr.cloud.MissingSegmentRecoveryTest.testLeaderRecovery(MissingSegmentRecoveryTest.java:105)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([9B90977827D05434:13C4A8A2892C39CC]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.testBasics(SharedFSAutoReplicaFailoverTest.java:307)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test(SharedFSAutoReplicaFailoverTest.java:136)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12210 lines...]
[junit4] Suite: org.apache.solr.cloud.MissingSegmentRecoveryTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.MissingSegmentRecoveryTest_9B90977827D05434-001/init-core-data-001
[junit4] 2> 640887 INFO (SUITE-MissingSegmentRecoveryTest-seed#[9B90977827D05434]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields
[junit4] 2> 640887 INFO (SUITE-MissingSegmentRecoveryTest-seed#[9B90977827D05434]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 640892 INFO (SUITE-MissingSegmentRecoveryTest-seed#[9B90977827D05434]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.MissingSegmentRecoveryTest_9B90977827D05434-001/tempDir-001
[junit4] 2> 640892 INFO (SUITE-MissingSegmentRecoveryTest-seed#[9B90977827D05434]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 640899 INFO (Thread-787) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 640899 INFO (Thread-787) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 640939 ERROR (Thread-787) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 640999 INFO (SUITE-MissingSegmentRecoveryTest-seed#[9B90977827D05434]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:54884
[junit4] 2> 641045 INFO (jetty-launcher-566-thread-1) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 641061 INFO (jetty-launcher-566-thread-2) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 641071 INFO (jetty-launcher-566-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@21f9f662{/solr,null,AVAILABLE}
[junit4] 2> 641073 INFO (jetty-launcher-566-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@33a91394{/solr,null,AVAILABLE}
[junit4] 2> 641074 INFO (jetty-launcher-566-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@766c7140{SSL,[ssl, http/1.1]}{127.0.0.1:35061}
[junit4] 2> 641074 INFO (jetty-launcher-566-thread-2) [ ] o.e.j.s.Server Started @643344ms
[junit4] 2> 641074 INFO (jetty-launcher-566-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35061}
[junit4] 2> 641074 ERROR (jetty-launcher-566-thread-2) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 641075 INFO (jetty-launcher-566-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.6.3
[junit4] 2> 641075 INFO (jetty-launcher-566-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 641075 INFO (jetty-launcher-566-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 641075 INFO (jetty-launcher-566-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-03-03T04:33:23.102Z
[junit4] 2> 641076 INFO (jetty-launcher-566-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@13ffe15b{SSL,[ssl, http/1.1]}{127.0.0.1:42561}
[junit4] 2> 641076 INFO (jetty-launcher-566-thread-1) [ ] o.e.j.s.Server Started @643345ms
[junit4] 2> 641076 INFO (jetty-launcher-566-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42561}
[junit4] 2> 641076 ERROR (jetty-launcher-566-thread-1) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 641076 INFO (jetty-launcher-566-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.6.3
[junit4] 2> 641076 INFO (jetty-launcher-566-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 641076 INFO (jetty-launcher-566-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 641076 INFO (jetty-launcher-566-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-03-03T04:33:23.103Z
[junit4] 2> 641078 INFO (jetty-launcher-566-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 641087 INFO (jetty-launcher-566-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 641095 INFO (jetty-launcher-566-thread-2) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 641096 INFO (jetty-launcher-566-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54884/solr
[junit4] 2> 641097 INFO (jetty-launcher-566-thread-1) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 641098 INFO (jetty-launcher-566-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54884/solr
[junit4] 2> 641110 INFO (jetty-launcher-566-thread-2) [n:127.0.0.1:35061_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 641110 INFO (jetty-launcher-566-thread-2) [n:127.0.0.1:35061_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:35061_solr
[junit4] 2> 641111 INFO (jetty-launcher-566-thread-2) [n:127.0.0.1:35061_solr ] o.a.s.c.Overseer Overseer (id=99618101850275845-127.0.0.1:35061_solr-n_0000000000) starting
[junit4] 2> 641117 INFO (jetty-launcher-566-thread-1) [n:127.0.0.1:42561_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 641117 INFO (jetty-launcher-566-thread-1) [n:127.0.0.1:42561_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42561_solr
[junit4] 2> 641362 INFO (jetty-launcher-566-thread-2) [n:127.0.0.1:35061_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35061_solr
[junit4] 2> 641367 INFO (zkCallback-578-thread-1-processing-n:127.0.0.1:42561_solr) [n:127.0.0.1:42561_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 641396 INFO (zkCallback-577-thread-1-processing-n:127.0.0.1:35061_solr) [n:127.0.0.1:35061_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 641397 INFO (OverseerStateUpdate-99618101850275845-127.0.0.1:35061_solr-n_0000000000) [n:127.0.0.1:35061_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 641408 INFO (zkCallback-578-thread-2-processing-n:127.0.0.1:42561_solr) [n:127.0.0.1:42561_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 641421 INFO (jetty-launcher-566-thread-2) [n:127.0.0.1:35061_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.MissingSegmentRecoveryTest_9B90977827D05434-001/tempDir-001/node2/.
[junit4] 2> 641443 INFO (jetty-launcher-566-thread-1) [n:127.0.0.1:42561_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.MissingSegmentRecoveryTest_9B90977827D05434-001/tempDir-001/node1/.
[junit4] 2> 641497 INFO (SUITE-MissingSegmentRecoveryTest-seed#[9B90977827D05434]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 641497 INFO (SUITE-MissingSegmentRecoveryTest-seed#[9B90977827D05434]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:54884/solr ready
[junit4] 2> 641518 INFO (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[9B90977827D05434]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testLeaderRecovery
[junit4] 2> 641593 INFO (qtp968564639-3268) [n:127.0.0.1:42561_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf&maxShardsPerNode=1&name=MissingSegmentRecoveryTest&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 641603 INFO (OverseerThreadFactory-1671-thread-1-processing-n:127.0.0.1:35061_solr) [n:127.0.0.1:35061_solr ] o.a.s.c.CreateCollectionCmd Create collection MissingSegmentRecoveryTest
[junit4] 2> 641753 INFO (qtp968564639-3263) [n:127.0.0.1:42561_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=MissingSegmentRecoveryTest_shard1_replica1&action=CREATE&numShards=1&collection=MissingSegmentRecoveryTest&shard=shard1&wt=javabin&version=2
[junit4] 2> 641754 INFO (qtp968564639-3263) [n:127.0.0.1:42561_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 641806 INFO (qtp228215942-3279) [n:127.0.0.1:35061_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=MissingSegmentRecoveryTest_shard1_replica2&action=CREATE&numShards=1&collection=MissingSegmentRecoveryTest&shard=shard1&wt=javabin&version=2
[junit4] 2> 641806 INFO (qtp228215942-3279) [n:127.0.0.1:35061_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 641909 INFO (zkCallback-577-thread-2-processing-n:127.0.0.1:35061_solr) [n:127.0.0.1:35061_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MissingSegmentRecoveryTest/state.json] for collection [MissingSegmentRecoveryTest] has occurred - updating... (live nodes size: [2])
[junit4] 2> 641909 INFO (zkCallback-578-thread-2-processing-n:127.0.0.1:42561_solr) [n:127.0.0.1:42561_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MissingSegmentRecoveryTest/state.json] for collection [MissingSegmentRecoveryTest] has occurred - updating... (live nodes size: [2])
[junit4] 2> 642770 INFO (qtp968564639-3263) [n:127.0.0.1:42561_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.3
[junit4] 2> 642777 INFO (qtp968564639-3263) [n:127.0.0.1:42561_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.s.IndexSchema [MissingSegmentRecoveryTest_shard1_replica1] Schema name=minimal
[junit4] 2> 642783 INFO (qtp968564639-3263) [n:127.0.0.1:42561_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 642783 INFO (qtp968564639-3263) [n:127.0.0.1:42561_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.c.CoreContainer Creating SolrCore 'MissingSegmentRecoveryTest_shard1_replica1' using configuration from collection MissingSegmentRecoveryTest, trusted=true
[junit4] 2> 642784 INFO (qtp968564639-3263) [n:127.0.0.1:42561_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 642784 INFO (qtp968564639-3263) [n:127.0.0.1:42561_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.c.SolrCore [[MissingSegmentRecoveryTest_shard1_replica1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.MissingSegmentRecoveryTest_9B90977827D05434-001/tempDir-001/node1/MissingSegmentRecoveryTest_shard1_replica1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.MissingSegmentRecoveryTest_9B90977827D05434-001/tempDir-001/node1/./MissingSegmentRecoveryTest_shard1_replica1/data/]
[junit4] 2> 642837 INFO (qtp228215942-3279) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.3
[junit4] 2> 642850 INFO (qtp228215942-3279) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.s.IndexSchema [MissingSegmentRecoveryTest_shard1_replica2] Schema name=minimal
[junit4] 2> 642852 INFO (qtp228215942-3279) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 642852 INFO (qtp228215942-3279) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.c.CoreContainer Creating SolrCore 'MissingSegmentRecoveryTest_shard1_replica2' using configuration from collection MissingSegmentRecoveryTest, trusted=true
[junit4] 2> 642852 INFO (qtp228215942-3279) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 642852 INFO (qtp228215942-3279) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.c.SolrCore [[MissingSegmentRecoveryTest_shard1_replica2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.MissingSegmentRecoveryTest_9B90977827D05434-001/tempDir-001/node2/MissingSegmentRecoveryTest_shard1_replica2], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.MissingSegmentRecoveryTest_9B90977827D05434-001/tempDir-001/node2/./MissingSegmentRecoveryTest_shard1_replica2/data/]
[junit4] 2> 642864 INFO (qtp968564639-3263) [n:127.0.0.1:42561_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 642864 INFO (qtp968564639-3263) [n:127.0.0.1:42561_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 642865 INFO (qtp968564639-3263) [n:127.0.0.1:42561_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 642865 INFO (qtp968564639-3263) [n:127.0.0.1:42561_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 642866 INFO (qtp968564639-3263) [n:127.0.0.1:42561_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@49c30950[MissingSegmentRecoveryTest_shard1_replica1] main]
[junit4] 2> 642867 INFO (qtp968564639-3263) [n:127.0.0.1:42561_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 642867 INFO (qtp968564639-3263) [n:127.0.0.1:42561_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 642868 INFO (qtp968564639-3263) [n:127.0.0.1:42561_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 643059 INFO (qtp968564639-3263) [n:127.0.0.1:42561_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1593889631854657536
[junit4] 2> 643063 INFO (searcherExecutor-1676-thread-1-processing-n:127.0.0.1:42561_solr x:MissingSegmentRecoveryTest_shard1_replica1 s:shard1 c:MissingSegmentRecoveryTest) [n:127.0.0.1:42561_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.c.SolrCore [MissingSegmentRecoveryTest_shard1_replica1] Registered new searcher Searcher@49c30950[MissingSegmentRecoveryTest_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 643069 INFO (qtp968564639-3263) [n:127.0.0.1:42561_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9999ms
[junit4] 2> 643115 INFO (qtp228215942-3279) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 643115 INFO (qtp228215942-3279) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 643129 INFO (qtp228215942-3279) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 643130 INFO (qtp228215942-3279) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 643131 INFO (qtp228215942-3279) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.s.SolrIndexSearcher Opening [Searcher@7f46291b[MissingSegmentRecoveryTest_shard1_replica2] main]
[junit4] 2> 643132 INFO (qtp228215942-3279) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 643132 INFO (qtp228215942-3279) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 643132 INFO (qtp228215942-3279) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 643132 INFO (qtp228215942-3279) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1593889631931203584
[junit4] 2> 643136 INFO (searcherExecutor-1677-thread-1-processing-n:127.0.0.1:35061_solr x:MissingSegmentRecoveryTest_shard1_replica2 s:shard1 c:MissingSegmentRecoveryTest) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.c.SolrCore [MissingSegmentRecoveryTest_shard1_replica2] Registered new searcher Searcher@7f46291b[MissingSegmentRecoveryTest_shard1_replica2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 643170 INFO (zkCallback-577-thread-2-processing-n:127.0.0.1:35061_solr) [n:127.0.0.1:35061_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MissingSegmentRecoveryTest/state.json] for collection [MissingSegmentRecoveryTest] has occurred - updating... (live nodes size: [2])
[junit4] 2> 643170 INFO (zkCallback-578-thread-2-processing-n:127.0.0.1:42561_solr) [n:127.0.0.1:42561_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MissingSegmentRecoveryTest/state.json] for collection [MissingSegmentRecoveryTest] has occurred - updating... (live nodes size: [2])
[junit4] 2> 643571 INFO (qtp968564639-3263) [n:127.0.0.1:42561_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 643571 INFO (qtp968564639-3263) [n:127.0.0.1:42561_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 643571 INFO (qtp968564639-3263) [n:127.0.0.1:42561_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:42561/solr/MissingSegmentRecoveryTest_shard1_replica1/
[junit4] 2> 643572 INFO (qtp968564639-3263) [n:127.0.0.1:42561_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.u.PeerSync PeerSync: core=MissingSegmentRecoveryTest_shard1_replica1 url=https://127.0.0.1:42561/solr START replicas=[https://127.0.0.1:35061/solr/MissingSegmentRecoveryTest_shard1_replica2/] nUpdates=100
[junit4] 2> 643575 INFO (qtp968564639-3263) [n:127.0.0.1:42561_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.u.PeerSync PeerSync: core=MissingSegmentRecoveryTest_shard1_replica1 url=https://127.0.0.1:42561/solr DONE. We have no versions. sync failed.
[junit4] 2> 643585 INFO (qtp228215942-3277) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.c.S.Request [MissingSegmentRecoveryTest_shard1_replica2] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 643586 INFO (qtp968564639-3263) [n:127.0.0.1:42561_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 643586 INFO (qtp968564639-3263) [n:127.0.0.1:42561_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 643586 INFO (qtp968564639-3263) [n:127.0.0.1:42561_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 643600 INFO (qtp968564639-3263) [n:127.0.0.1:42561_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:42561/solr/MissingSegmentRecoveryTest_shard1_replica1/ shard1
[junit4] 2> 643702 INFO (zkCallback-577-thread-2-processing-n:127.0.0.1:35061_solr) [n:127.0.0.1:35061_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MissingSegmentRecoveryTest/state.json] for collection [MissingSegmentRecoveryTest] has occurred - updating... (live nodes size: [2])
[junit4] 2> 643702 INFO (zkCallback-578-thread-2-processing-n:127.0.0.1:42561_solr) [n:127.0.0.1:42561_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MissingSegmentRecoveryTest/state.json] for collection [MissingSegmentRecoveryTest] has occurred - updating... (live nodes size: [2])
[junit4] 2> 643751 INFO (qtp968564639-3263) [n:127.0.0.1:42561_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 643753 INFO (qtp968564639-3263) [n:127.0.0.1:42561_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=MissingSegmentRecoveryTest_shard1_replica1&action=CREATE&numShards=1&collection=MissingSegmentRecoveryTest&shard=shard1&wt=javabin&version=2} status=0 QTime=1999
[junit4] 2> 643854 INFO (zkCallback-578-thread-2-processing-n:127.0.0.1:42561_solr) [n:127.0.0.1:42561_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MissingSegmentRecoveryTest/state.json] for collection [MissingSegmentRecoveryTest] has occurred - updating... (live nodes size: [2])
[junit4] 2> 643854 INFO (zkCallback-577-thread-2-processing-n:127.0.0.1:35061_solr) [n:127.0.0.1:35061_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MissingSegmentRecoveryTest/state.json] for collection [MissingSegmentRecoveryTest] has occurred - updating... (live nodes size: [2])
[junit4] 2> 644138 INFO (qtp228215942-3279) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=MissingSegmentRecoveryTest_shard1_replica2&action=CREATE&numShards=1&collection=MissingSegmentRecoveryTest&shard=shard1&wt=javabin&version=2} status=0 QTime=2332
[junit4] 2> 644142 INFO (qtp968564639-3268) [n:127.0.0.1:42561_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 644244 INFO (zkCallback-578-thread-2-processing-n:127.0.0.1:42561_solr) [n:127.0.0.1:42561_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MissingSegmentRecoveryTest/state.json] for collection [MissingSegmentRecoveryTest] has occurred - updating... (live nodes size: [2])
[junit4] 2> 644244 INFO (zkCallback-577-thread-2-processing-n:127.0.0.1:35061_solr) [n:127.0.0.1:35061_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MissingSegmentRecoveryTest/state.json] for collection [MissingSegmentRecoveryTest] has occurred - updating... (live nodes size: [2])
[junit4] 2> 645142 INFO (qtp968564639-3268) [n:127.0.0.1:42561_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=2&collection.configName=conf&maxShardsPerNode=1&name=MissingSegmentRecoveryTest&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=3549
[junit4] 2> 645166 INFO (qtp228215942-3273) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [MissingSegmentRecoveryTest_shard1_replica2] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:42561/solr/MissingSegmentRecoveryTest_shard1_replica1/&wt=javabin&version=2}{add=[0 (1593889634056667136), 1 (1593889634058764288), 2 (1593889634058764289), 3 (1593889634058764290), 4 (1593889634058764291), 5 (1593889634058764292), 6 (1593889634058764293), 7 (1593889634058764294), 8 (1593889634058764295), 9 (1593889634058764296)]} 0 1
[junit4] 2> 645166 INFO (qtp968564639-3269) [n:127.0.0.1:42561_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node1 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [MissingSegmentRecoveryTest_shard1_replica1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[0 (1593889634056667136), 1 (1593889634058764288), 2 (1593889634058764289), 3 (1593889634058764290), 4 (1593889634058764291), 5 (1593889634058764292), 6 (1593889634058764293), 7 (1593889634058764294), 8 (1593889634058764295), 9 (1593889634058764296)]} 0 7
[junit4] 2> 645180 INFO (qtp228215942-3276) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 645180 INFO (qtp228215942-3276) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@250c1752
[junit4] 2> 645196 INFO (qtp968564639-3263) [n:127.0.0.1:42561_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node1 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 645196 INFO (qtp968564639-3263) [n:127.0.0.1:42561_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node1 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@257d07d2
[junit4] 2> 645231 INFO (qtp968564639-3263) [n:127.0.0.1:42561_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node1 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4aed6040[MissingSegmentRecoveryTest_shard1_replica1] main]
[junit4] 2> 645232 INFO (qtp968564639-3263) [n:127.0.0.1:42561_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node1 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 645232 INFO (searcherExecutor-1676-thread-1-processing-n:127.0.0.1:42561_solr x:MissingSegmentRecoveryTest_shard1_replica1 s:shard1 c:MissingSegmentRecoveryTest r:core_node1) [n:127.0.0.1:42561_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node1 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.c.SolrCore [MissingSegmentRecoveryTest_shard1_replica1] Registered new searcher Searcher@4aed6040[MissingSegmentRecoveryTest_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(6.6.3):C10)))}
[junit4] 2> 645232 INFO (qtp228215942-3276) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.s.SolrIndexSearcher Opening [Searcher@e97669f[MissingSegmentRecoveryTest_shard1_replica2] main]
[junit4] 2> 645233 INFO (qtp228215942-3276) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 645233 INFO (searcherExecutor-1677-thread-1-processing-n:127.0.0.1:35061_solr x:MissingSegmentRecoveryTest_shard1_replica2 s:shard1 c:MissingSegmentRecoveryTest r:core_node2) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.c.SolrCore [MissingSegmentRecoveryTest_shard1_replica2] Registered new searcher Searcher@e97669f[MissingSegmentRecoveryTest_shard1_replica2] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(6.6.3):C10)))}
[junit4] 2> 645233 INFO (qtp228215942-3276) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.u.p.LogUpdateProcessorFactory [MissingSegmentRecoveryTest_shard1_replica2] webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=https://127.0.0.1:42561/solr/MissingSegmentRecoveryTest_shard1_replica1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 53
[junit4] 2> 645233 INFO (qtp968564639-3263) [n:127.0.0.1:42561_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node1 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [MissingSegmentRecoveryTest_shard1_replica1] webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=https://127.0.0.1:42561/solr/MissingSegmentRecoveryTest_shard1_replica1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 36
[junit4] 2> 645234 INFO (qtp968564639-3267) [n:127.0.0.1:42561_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node1 x:MissingSegmentRecoveryTest_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [MissingSegmentRecoveryTest_shard1_replica1] webapp=/solr path=/update params={_stateVer_=MissingSegmentRecoveryTest:5&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 66
[junit4] 2> 645247 INFO (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[9B90977827D05434]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@766c7140{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 645263 INFO (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[9B90977827D05434]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=934350477
[junit4] 2> 645263 INFO (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[9B90977827D05434]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.node
[junit4] 2> 645263 INFO (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[9B90977827D05434]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.jvm
[junit4] 2> 645263 INFO (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[9B90977827D05434]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.jetty
[junit4] 2> 645275 INFO (coreCloseExecutor-1686-thread-1) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.c.SolrCore [MissingSegmentRecoveryTest_shard1_replica2] CLOSING SolrCore org.apache.solr.core.SolrCore@694ba92f
[junit4] 2> 645292 INFO (coreCloseExecutor-1686-thread-1) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.MissingSegmentRecoveryTest.shard1.replica2
[junit4] 2> 645294 INFO (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[9B90977827D05434]) [ ] o.a.s.c.Overseer Overseer (id=99618101850275845-127.0.0.1:35061_solr-n_0000000000) closing
[junit4] 2> 645294 INFO (OverseerStateUpdate-99618101850275845-127.0.0.1:35061_solr-n_0000000000) [n:127.0.0.1:35061_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:35061_solr
[junit4] 2> 645297 INFO (zkCallback-584-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 645297 INFO (zkCallback-578-thread-1-processing-n:127.0.0.1:42561_solr) [n:127.0.0.1:42561_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 645297 INFO (zkCallback-578-thread-2-processing-n:127.0.0.1:42561_solr) [n:127.0.0.1:42561_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42561_solr
[junit4] 2> 645297 INFO (zkCallback-578-thread-2-processing-n:127.0.0.1:42561_solr) [n:127.0.0.1:42561_solr ] o.a.s.c.Overseer Overseer (id=99618101850275846-127.0.0.1:42561_solr-n_0000000001) starting
[junit4] 2> 645311 INFO (zkCallback-578-thread-2-processing-n:127.0.0.1:42561_solr) [n:127.0.0.1:42561_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MissingSegmentRecoveryTest/state.json] for collection [MissingSegmentRecoveryTest] has occurred - updating... (live nodes size: [1])
[junit4] 2> 646797 WARN (zkCallback-577-thread-2-processing-n:127.0.0.1:35061_solr) [n:127.0.0.1:35061_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 646798 INFO (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[9B90977827D05434]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@33a91394{/solr,null,UNAVAILABLE}
[junit4] 2> 646801 INFO (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[9B90977827D05434]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 646802 INFO (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[9B90977827D05434]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3a008140{/solr,null,AVAILABLE}
[junit4] 2> 646804 INFO (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[9B90977827D05434]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@304ee53e{SSL,[ssl, http/1.1]}{127.0.0.1:35061}
[junit4] 2> 646804 INFO (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[9B90977827D05434]) [ ] o.e.j.s.Server Started @649073ms
[junit4] 2> 646804 INFO (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[9B90977827D05434]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35061}
[junit4] 2> 646804 ERROR (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[9B90977827D05434]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 646805 INFO (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[9B90977827D05434]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.6.3
[junit4] 2> 646805 INFO (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[9B90977827D05434]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 646805 INFO (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[9B90977827D05434]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 646805 INFO (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[9B90977827D05434]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-03-03T04:33:28.832Z
[junit4] 2> 646814 INFO (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[9B90977827D05434]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 646820 INFO (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[9B90977827D05434]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 646828 INFO (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[9B90977827D05434]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54884/solr
[junit4] 2> 646835 INFO (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[9B90977827D05434]) [n:127.0.0.1:35061_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 646836 INFO (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[9B90977827D05434]) [n:127.0.0.1:35061_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 646837 INFO (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[9B90977827D05434]) [n:127.0.0.1:35061_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35061_solr
[junit4] 2> 646838 INFO (zkCallback-584-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 646838 INFO (zkCallback-578-thread-2-processing-n:127.0.0.1:42561_solr) [n:127.0.0.1:42561_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 646845 INFO (zkCallback-589-thread-1-processing-n:127.0.0.1:35061_solr) [n:127.0.0.1:35061_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 646903 INFO (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[9B90977827D05434]) [n:127.0.0.1:35061_solr ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.MissingSegmentRecoveryTest_9B90977827D05434-001/tempDir-001/node2/.
[junit4] 2> 646904 INFO (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[9B90977827D05434]) [n:127.0.0.1:35061_solr ] o.a.s.c.CorePropertiesLocator Cores are: [MissingSegmentRecoveryTest_shard1_replica2]
[junit4] 2> 646905 INFO (coreLoadExecutor-1695-thread-1-processing-n:127.0.0.1:35061_solr) [n:127.0.0.1:35061_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 646924 INFO (coreLoadExecutor-1695-thread-1-processing-n:127.0.0.1:35061_solr) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.3
[junit4] 2> 646932 INFO (coreLoadExecutor-1695-thread-1-processing-n:127.0.0.1:35061_solr) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.s.IndexSchema [MissingSegmentRecoveryTest_shard1_replica2] Schema name=minimal
[junit4] 2> 646934 INFO (coreLoadExecutor-1695-thread-1-processing-n:127.0.0.1:35061_solr) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 646934 INFO (coreLoadExecutor-1695-thread-1-processing-n:127.0.0.1:35061_solr) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.c.CoreContainer Creating SolrCore 'MissingSegmentRecoveryTest_shard1_replica2' using configuration from collection MissingSegmentRecoveryTest, trusted=true
[junit4] 2> 646934 INFO (coreLoadExecutor-1695-thread-1-processing-n:127.0.0.1:35061_solr) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 646934 INFO (coreLoadExecutor-1695-thread-1-processing-n:127.0.0.1:35061_solr) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.c.SolrCore [[MissingSegmentRecoveryTest_shard1_replica2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.MissingSegmentRecoveryTest_9B90977827D05434-001/tempDir-001/node2/MissingSegmentRecoveryTest_shard1_replica2], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.MissingSegmentRecoveryTest_9B90977827D05434-001/tempDir-001/node2/./MissingSegmentRecoveryTest_shard1_replica2/data/]
[junit4] 2> 646961 INFO (coreLoadExecutor-1695-thread-1-processing-n:127.0.0.1:35061_solr) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 646961 INFO (coreLoadExecutor-1695-thread-1-processing-n:127.0.0.1:35061_solr) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 646962 INFO (coreLoadExecutor-1695-thread-1-processing-n:127.0.0.1:35061_solr) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 646962 INFO (coreLoadExecutor-1695-thread-1-processing-n:127.0.0.1:35061_solr) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 646964 INFO (coreLoadExecutor-1695-thread-1-processing-n:127.0.0.1:35061_solr) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.c.SolrCore [MissingSegmentRecoveryTest_shard1_replica2] CLOSING SolrCore org.apache.solr.core.SolrCore@7bd72a33
[junit4] 2> 646964 INFO (coreLoadExecutor-1695-thread-1-processing-n:127.0.0.1:35061_solr) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.MissingSegmentRecoveryTest.shard1.replica2
[junit4] 2> 646965 INFO (coreLoadExecutor-1695-thread-1-processing-n:127.0.0.1:35061_solr) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.c.CoreContainer Found active leader, will attempt to create fresh core and recover.
[junit4] 2> 646966 INFO (coreLoadExecutor-1695-thread-1-processing-n:127.0.0.1:35061_solr) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.c.SolrCore Updating index properties... index=index.20180303013328993
[junit4] 2> 646970 INFO (coreLoadExecutor-1695-thread-1-processing-n:127.0.0.1:35061_solr) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 646970 INFO (coreLoadExecutor-1695-thread-1-processing-n:127.0.0.1:35061_solr) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.c.SolrCore [[MissingSegmentRecoveryTest_shard1_replica2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.MissingSegmentRecoveryTest_9B90977827D05434-001/tempDir-001/node2/MissingSegmentRecoveryTest_shard1_replica2], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.MissingSegmentRecoveryTest_9B90977827D05434-001/tempDir-001/node2/./MissingSegmentRecoveryTest_shard1_replica2/data/]
[junit4] 2> 646975 INFO (OldIndexDirectoryCleanupThreadForCore-MissingSegmentRecoveryTest_shard1_replica2) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.c.DirectoryFactory Found 1 old index directories to clean-up under /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.MissingSegmentRecoveryTest_9B90977827D05434-001/tempDir-001/node2/./MissingSegmentRecoveryTest_shard1_replica2/data/ afterReload=false
[junit4] 2> 646979 INFO (OldIndexDirectoryCleanupThreadForCore-MissingSegmentRecoveryTest_shard1_replica2) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.c.DirectoryFactory Deleted old index directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.MissingSegmentRecoveryTest_9B90977827D05434-001/tempDir-001/node2/./MissingSegmentRecoveryTest_shard1_replica2/data/index.20180303013328993
[junit4] 2> 646980 ERROR (coreLoadExecutor-1695-thread-1-processing-n:127.0.0.1:35061_solr) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.u.SolrIndexWriter Error closing IndexWriter
[junit4] 2> java.nio.file.NoSuchFileException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.MissingSegmentRecoveryTest_9B90977827D05434-001/tempDir-001/node2/MissingSegmentRecoveryTest_shard1_replica2/data/index.20180303013328993
[junit4] 2> at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
[junit4] 2> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
[junit4] 2> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
[junit4] 2> at sun.nio.fs.UnixFileSystemProvider.newDirectoryStream(UnixFileSystemProvider.java:427)
[junit4] 2> at java.nio.file.Files.newDirectoryStream(Files.java:457)
[junit4] 2> at org.apache.lucene.store.FSDirectory.listAll(FSDirectory.java:215)
[junit4] 2> at org.apache.lucene.store.FSDirectory.listAll(FSDirectory.java:234)
[junit4] 2> at org.apache.lucene.store.FilterDirectory.listAll(FilterDirectory.java:57)
[junit4] 2> at org.apache.lucene.index.IndexFileDeleter.refresh(IndexFileDeleter.java:429)
[junit4] 2> at org.apache.lucene.index.IndexWriter.rollbackInternalNoCommit(IndexWriter.java:2249)
[junit4] 2> at org.apache.lucene.index.IndexWriter.rollbackInternal(IndexWriter.java:2193)
[junit4] 2> at org.apache.lucene.index.IndexWriter.shutdown(IndexWriter.java:1129)
[junit4] 2> at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1171)
[junit4] 2> at org.apache.solr.update.SolrIndexWriter.close(SolrIndexWriter.java:278)
[junit4] 2> at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:730)
[junit4] 2> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:923)
[junit4] 2> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:830)
[junit4] 2> at org.apache.solr.core.CoreContainer.processCoreCreateException(CoreContainer.java:1026)
[junit4] 2> at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:952)
[junit4] 2> at org.apache.solr.core.CoreContainer.lambda$load$5(CoreContainer.java:565)
[junit4] 2> at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 647003 INFO (OldIndexDirectoryCleanupThreadForCore-MissingSegmentRecoveryTest_shard1_replica2) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.c.DirectoryFactory Found 1 old index directories to clean-up under /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.MissingSegmentRecoveryTest_9B90977827D05434-001/tempDir-001/node2/./MissingSegmentRecoveryTest_shard1_replica2/data/ afterReload=false
[junit4] 2> 647022 INFO (zkCallback-578-thread-2-processing-n:127.0.0.1:42561_solr) [n:127.0.0.1:42561_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MissingSegmentRecoveryTest/state.json] for collection [MissingSegmentRecoveryTest] has occurred - updating... (live nodes size: [2])
[junit4] 2> 647022 INFO (OldIndexDirectoryCleanupThreadForCore-MissingSegmentRecoveryTest_shard1_replica2) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.c.DirectoryFactory Deleted old index directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.MissingSegmentRecoveryTest_9B90977827D05434-001/tempDir-001/node2/./MissingSegmentRecoveryTest_shard1_replica2/data/index
[junit4] 2> 647023 INFO (zkCallback-589-thread-1-processing-n:127.0.0.1:35061_solr) [n:127.0.0.1:35061_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MissingSegmentRecoveryTest/state.json] for collection [MissingSegmentRecoveryTest] has occurred - updating... (live nodes size: [2])
[junit4] 2> 647033 INFO (coreLoadExecutor-1695-thread-1-processing-n:127.0.0.1:35061_solr) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 647033 INFO (coreLoadExecutor-1695-thread-1-processing-n:127.0.0.1:35061_solr) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 647034 INFO (coreLoadExecutor-1695-thread-1-processing-n:127.0.0.1:35061_solr) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 647034 INFO (coreLoadExecutor-1695-thread-1-processing-n:127.0.0.1:35061_solr) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 647035 INFO (coreLoadExecutor-1695-thread-1-processing-n:127.0.0.1:35061_solr) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.c.SolrCore [MissingSegmentRecoveryTest_shard1_replica2] CLOSING SolrCore org.apache.solr.core.SolrCore@5b100ec
[junit4] 2> 647036 INFO (coreLoadExecutor-1695-thread-1-processing-n:127.0.0.1:35061_solr) [n:127.0.0.1:35061_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node2 x:MissingSegmentRecoveryTest_shard1_replica2] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.MissingSegmentRecoveryTest.shard1.replica2
[junit4] 2> 647036 ERROR (coreContainerWorkExecutor-1691-thread-1-processing-n:127.0.0.1:35061_solr) [n:127.0.0.1:35061_solr ] o.a.s.c.CoreContainer Error waiting for SolrCore to be created
[junit4] 2> java.util.concurrent.ExecutionException: org.apache.solr.common.SolrException: Unable to create core [MissingSegmentRecoveryTest_shard1_replica2]
[junit4] 2> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
[junit4] 2> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
[junit4] 2> at org.apache.solr.core.CoreContainer.lambda$load$6(CoreContainer.java:594)
[junit4] 2> at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: org.apache.solr.common.SolrException: Unable to create core [MissingSegmentRecoveryTest_shard1_replica2]
[junit4] 2> at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:966)
[junit4] 2> at org.apache.solr.core.CoreContainer.lambda$load$5(CoreContainer.java:565)
[junit4] 2> at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
[junit4] 2> ... 5 more
[junit4] 2> Caused by: org.apache.solr.common.SolrException: Error opening new searcher
[junit4] 2> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:977)
[junit4] 2> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:830)
[junit4] 2> at org.apache.solr.core.CoreContainer.processCoreCreateException(CoreContainer.java:1026)
[junit4] 2> at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:952)
[junit4] 2> ... 7 more
[junit4] 2> Suppressed: org.apache.solr.common.SolrException: Error opening new searcher
[junit4] 2> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:977)
[junit4] 2> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:830)
[junit4] 2> at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:950)
[junit4] 2> ... 7 more
[junit4] 2> Caused by: org.apache.solr.common.SolrException: Error opening new searcher
[junit4] 2> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:2069)
[junit4] 2> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:2189)
[junit4] 2> at org.apache.solr.core.SolrCore.initSearcher(SolrCore.java:1071)
[junit4] 2> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:949)
[junit4] 2> ... 9 more
[junit4] 2> Caused by: org.apache.lucene.index.CorruptIndexException: Unexpected file read error while reading index. (resource=BufferedChecksumIndexInput(MMapIndexInput(path="/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.MissingSegmentRecoveryTest_9B90977827D05434-001/tempDir-001/node2/MissingSegmentRecoveryTest_shard1_replica2/data/index/segments_2")))
[junit4] 2> at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:290)
[junit4] 2> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:930)
[junit4] 2> at org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:118)
[junit4] 2> at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:93)
[junit4] 2> at org.apache.solr.update.DefaultSolrCoreState.createMainIndexWriter(DefaultSolrCoreState.java:257)
[junit4] 2> at org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:131)
[junit4] 2> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:2030)
[junit4] 2> ... 12 more
[junit4] 2> Caused by: java.io.EOFException: read past EOF: MMapIndexInput(path="/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.MissingSegmentRecoveryTest_9B90977827D05434-001/tempDir-001/node2/MissingSegmentRecoveryTest_shard1_replica2/data/index/segments_2")
[junit4] 2> at org.apache.lucene.store.ByteBufferIndexInput.readByte(ByteBufferIndexInput.java:75)
[junit4] 2> at org.apache.lucene.store.BufferedChecksumIndexInput.readByte(BufferedChecksumIndexInput.java:41)
[junit4] 2> at org.apache.lucene.store.DataInput.readInt(DataInput.java:101)
[junit4] 2> at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:300)
[junit4] 2> at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:288)
[junit4] 2> ... 18 more
[junit4] 2> Caused by: org.apache.solr.common.SolrException: Error opening new searcher
[junit4] 2> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:2069)
[junit4] 2> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:2189)
[junit4] 2> at org.apache.solr.core.SolrCore.initSearcher(SolrCore.java:1071)
[junit4] 2> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:949)
[junit4] 2> ... 10 more
[junit4] 2> Caused by: org.apache.lucene.index.IndexNotFoundException: no segments* file found in LockValidatingDirectoryWrapper(MMapDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.MissingSegmentRecoveryTest_9B90977827D05434-001/tempDir-001/node2/MissingSegmentRecoveryTest_shard1_replica2/data/index.20180303013328993 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6e4c7412): files: [write.lock]
[junit4] 2> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:925)
[junit4] 2> at org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:118)
[junit4] 2> at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:93)
[junit4] 2> at org.apache.solr.update.DefaultSolrCoreState.createMainIndexWriter(DefaultSolrCoreState.java:257)
[junit4] 2> at org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:131)
[junit4] 2> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:2030)
[junit4] 2> ... 13 more
[junit4] 2> 737053 INFO (qtp968564639-3268) [n:127.0.0.1:42561_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params name=MissingSegmentRecoveryTest&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 737057 INFO (OverseerThreadFactory-1689-thread-1-processing-n:127.0.0.1:42561_solr) [n:127.0.0.1:42561_solr ] o.a.s.c.OverseerCollectionMessageHandler Executing Collection Cmd : action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true
[junit4] 2> 737061 INFO (qtp968564639-3267) [n:127.0.0.1:42561_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.MissingSegmentRecoveryTest.shard1.replica1
[junit4] 2> 737061 INFO (qtp968564639-3267) [n:127.0.0.1:42561_solr ] o.a.s.c.SolrCore [MissingSegmentRecoveryTest_shard1_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@2e4857ee
[junit4] 2> 737068 INFO (qtp968564639-3267) [n:127.0.0.1:42561_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.MissingSegmentRecoveryTest.shard1.replica1
[junit4] 2> 737084 INFO (qtp968564639-3267) [n:127.0.0.1:42561_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=MissingSegmentRecoveryTest_shard1_replica1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=22
[junit4] 2> 737088 INFO (zkCallback-589-thread-2-processing-n:127.0.0.1:35061_solr) [n:127.0.0.1:35061_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MissingSegmentRecoveryTest/state.json] for collection [MissingSegmentRecoveryTest] has occurred - updating... (live nodes size: [2])
[junit4] 2> 737126 INFO (qtp2111350895-3350) [n:127.0.0.1:35061_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=MissingSegmentRecoveryTest_shard1_replica2&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 737233 INFO (zkCallback-589-thread-2-processing-n:127.0.0.1:35061_solr) [n:127.0.0.1:35061_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/MissingSegmentRecoveryTest/state.json] for collection [MissingSegmentRecoveryTest] has occurred - updating... (live nodes size: [2])
[junit4] 2> 737834 INFO (qtp968564639-3268) [n:127.0.0.1:42561_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=MissingSegmentRecoveryTest&action=DELETE&wt=javabin&version=2} status=0 QTime=780
[junit4] 2> 737834 INFO (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[9B90977827D05434]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testLeaderRecovery
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=MissingSegmentRecoveryTest -Dtests.method=testLeaderRecovery -Dtests.seed=9B90977827D05434 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/test-data/enwiki.random.lines.txt -Dtests.locale=el-CY -Dtests.timezone=America/Argentina/San_Luis -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] FAILURE 96.3s J1 | MissingSegmentRecoveryTest.testLeaderRecovery <<<
[junit4] > Throwable #1: java.lang.AssertionError: Expected a collection with one shard and two replicas
[junit4] > null
[junit4] > Last available state: DocCollection(MissingSegmentRecoveryTest//collections/MissingSegmentRecoveryTest/state.json/7)={
[junit4] > "replicationFactor":"2",
[junit4] > "shards":{"shard1":{
[junit4] > "range":"80000000-7fffffff",
[junit4] > "state":"active",
[junit4] > "replicas":{
[junit4] > "core_node1":{
[junit4] > "core":"MissingSegmentRecoveryTest_shard1_replica1",
[junit4] > "base_url":"https://127.0.0.1:42561/solr",
[junit4] > "node_name":"127.0.0.1:42561_solr",
[junit4] > "state":"active",
[junit4] > "leader":"true"},
[junit4] > "core_node2":{
[junit4] > "core":"MissingSegmentRecoveryTest_shard1_replica2",
[junit4] > "base_url":"https://127.0.0.1:35061/solr",
[junit4] > "node_name":"127.0.0.1:35061_solr",
[junit4] > "state":"down"}}}},
[junit4] > "router":{"name":"compositeId"},
[junit4] > "maxShardsPerNode":"1",
[junit4] > "autoAddReplicas":"false"}
[junit4] > at __randomizedtesting.SeedInfo.seed([9B90977827D05434:CBC50F7B7EF1E229]:0)
[junit4] > at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:265)
[junit4] > at org.apache.solr.cloud.MissingSegmentRecoveryTest.testLeaderRecovery(MissingSegmentRecoveryTest.java:105)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 737840 INFO (jetty-closer-567-thread-1) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@304ee53e{SSL,[ssl, http/1.1]}{127.0.0.1:35061}
[junit4] 2> 737840 INFO (jetty-closer-567-thread-1) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1792811725
[junit4] 2> 737841 INFO (jetty-closer-567-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.node
[junit4] 2> 737841 INFO (jetty-closer-567-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.jvm
[junit4] 2> 737841 INFO (jetty-closer-567-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.jetty
[junit4] 2> 737841 INFO (jetty-closer-567-thread-2) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@13ffe15b{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 737841 INFO (jetty-closer-567-thread-2) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=858018595
[junit4] 2> 737841 INFO (jetty-closer-567-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.node
[junit4] 2> 737841 INFO (jetty-closer-567-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.jvm
[junit4] 2> 737842 INFO (jetty-closer-567-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.jetty
[junit4] 2> 737848 INFO (zkCallback-578-thread-3-processing-n:127.0.0.1:42561_solr) [n:127.0.0.1:42561_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 737849 INFO (jetty-closer-567-thread-2) [ ] o.a.s.c.Overseer Overseer (id=99618101850275846-127.0.0.1:42561_solr-n_0000000001) closing
[junit4] 2> 737850 INFO (OverseerStateUpdate-99618101850275846-127.0.0.1:42561_solr-n_0000000001) [n:127.0.0.1:42561_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:42561_solr
[junit4] 2> 737851 WARN (zkCallback-578-thread-4-processing-n:127.0.0.1:42561_solr) [n:127.0.0.1:42561_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 737852 INFO (jetty-closer-567-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@21f9f662{/solr,null,UNAVAILABLE}
[junit4] 2> 739348 WARN (zkCallback-589-thread-2-processing-n:127.0.0.1:35061_solr) [n:127.0.0.1:35061_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 739349 INFO (jetty-closer-567-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@3a008140{/solr,null,UNAVAILABLE}
[junit4] 2> 739350 ERROR (SUITE-MissingSegmentRecoveryTest-seed#[9B90977827D05434]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 739351 INFO (SUITE-MissingSegmentRecoveryTest-seed#[9B90977827D05434]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:54884 54884
[junit4] 2> 739530 INFO (Thread-787) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:54884 54884
[junit4] 2> 739531 WARN (Thread-787) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 4 /solr/aliases.json
[junit4] 2> 3 /solr/security.json
[junit4] 2> 2 /solr/configs/conf
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 4 /solr/clusterstate.json
[junit4] 2> 4 /solr/collections/MissingSegmentRecoveryTest/state.json
[junit4] 2> 4 /solr/clusterprops.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 4 /solr/collections
[junit4] 2> 3 /solr/live_nodes
[junit4] 2> 2 /solr/overseer/queue
[junit4] 2> 2 /solr/overseer/collection-queue-work
[junit4] 2>
[junit4] 2> 739531 INFO (SUITE-MissingSegmentRecoveryTest-seed#[9B90977827D05434]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.MissingSegmentRecoveryTest_9B90977827D05434-001
[junit4] 2> Mar 03, 2018 4:35:01 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): {_version_=BlockTreeOrds(blocksize=128), id=PostingsFormat(name=Memory doPackFST= true)}, docValues:{}, maxPointsInLeafNode=816, maxMBSortInHeap=5.423348067235001, sim=RandomSimilarity(queryNorm=true,coord=yes): {}, locale=el-CY, timezone=America/Argentina/San_Luis
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=67306928,total=531628032
[junit4] 2> NOTE: All tests run in this JVM: [TestSolrCloudSnapshots, MoreLikeThisHandlerTest, TestRandomFlRTGCloud, TestSolrCloudWithSecureImpersonation, TestFastOutputStream, BasicFunctionalityTest, TestReloadAndDeleteDocs, IndexBasedSpellCheckerTest, FileUtilsTest, TestSizeLimitedDistributedMap, TestCoreContainer, TestSearchPerf, ChaosMonkeySafeLeaderTest, TestBlendedInfixSuggestions, TestPointFields, CollectionsAPISolrJTest, TestMinMaxOnMultiValuedField, TestManagedSchema, TestMacros, DocValuesTest, BadCopyFieldTest, MissingSegmentRecoveryTest]
[junit4] Completed [116/714 (1!)] on J1 in 99.19s, 1 test, 1 failure <<< FAILURES!
[...truncated 93 lines...]
[junit4] Suite: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_9B90977827D05434-001/init-core-data-001
[junit4] 2> 741039 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9B90977827D05434]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=16 numCloses=16
[junit4] 2> 741039 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9B90977827D05434]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields
[junit4] 2> 741040 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9B90977827D05434]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=None)
[junit4] 2> 741040 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9B90977827D05434]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /_he/qw
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 741910 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9B90977827D05434]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 742127 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9B90977827D05434]-worker) [ ] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
[junit4] 2> 742144 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9B90977827D05434]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 742200 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9B90977827D05434]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 742243 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9B90977827D05434]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_34039_hdfs____.7qm30h/webapp
[junit4] 2> 748386 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9B90977827D05434]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34039
[junit4] 2> 749134 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9B90977827D05434]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 749138 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9B90977827D05434]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 749157 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9B90977827D05434]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_60297_datanode____.1g624j/webapp
[junit4] 2> 750681 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9B90977827D05434]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:60297
[junit4] 2> 750980 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9B90977827D05434]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 750981 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9B90977827D05434]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 750997 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9B90977827D05434]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_50359_datanode____oua3k3/webapp
[junit4] 2> 751506 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9B90977827D05434]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:50359
[junit4] 2> 751745 ERROR (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_9B90977827D05434-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_9B90977827D05434-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to localhost/127.0.0.1:44381) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 751805 ERROR (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_9B90977827D05434-001/tempDir-001/hdfsBaseDir/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_9B90977827D05434-001/tempDir-001/hdfsBaseDir/data/data4/]] heartbeating to localhost/127.0.0.1:44381) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 751934 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x482208b6e4a526: from storage DS-eab6854f-eb25-4268-bea0-79217409eda2 node DatanodeRegistration(127.0.0.1:50213, datanodeUuid=6a562b5d-a072-43a4-97e3-4a80b6c44971, infoPort=33173, infoSecurePort=0, ipcPort=51085, storageInfo=lv=-56;cid=testClusterID;nsid=428486816;c=0), blocks: 0, hasStaleStorage: true, processing time: 4 msecs
[junit4] 2> 751934 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x482208b6fcfe37: from storage DS-930c0b2c-1f48-4966-b1e9-eb403480c793 node DatanodeRegistration(127.0.0.1:44050, datanodeUuid=bab998d6-c809-4db4-8e14-6e9b79b0a53e, infoPort=58299, infoSecurePort=0, ipcPort=58930, storageInfo=lv=-56;cid=testClusterID;nsid=428486816;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 751934 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x482208b6e4a526:
[...truncated too long message...]
hdfs_home/solrj_collection/core_node2/data
[junit4] 2> 945873 WARN (zkCallback-644-thread-3-processing-n:127.0.0.1:35226__he%2Fqw) [n:127.0.0.1:35226__he%2Fqw ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 945874 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9B90977827D05434]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@11f1ec0b{HTTP/1.1,[http/1.1]}{127.0.0.1:35226}
[junit4] 2> 945874 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9B90977827D05434]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@255b5eb5{/_he/qw,null,UNAVAILABLE}
[junit4] 2> 945875 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9B90977827D05434]) [ ] o.a.s.c.ChaosMonkey monkey: stop shard! 60627
[junit4] 2> 945875 ERROR (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9B90977827D05434]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 945876 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9B90977827D05434]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:60154 60154
[junit4] 2> 946144 INFO (Thread-927) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:60154 60154
[junit4] 2> 946146 WARN (Thread-927) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 12 /solr/aliases.json
[junit4] 2> 10 /solr/security.json
[junit4] 2> 10 /solr/configs/conf1
[junit4] 2> 6 /solr/clusterprops.json
[junit4] 2> 4 /solr/collections/collection1/state.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 96 /solr/collections/solrj_collection2/state.json
[junit4] 2> 96 /solr/collections/solrj_collection3/state.json
[junit4] 2> 93 /solr/collections/solrj_collection/state.json
[junit4] 2> 80 /solr/collections/solrj_collection4/state.json
[junit4] 2> 12 /solr/clusterprops.json
[junit4] 2> 12 /solr/clusterstate.json
[junit4] 2> 3 /solr/overseer_elect/election/99618109152231432-127.0.0.1:35149__he%2Fqw-n_0000000001
[junit4] 2> 2 /solr/collections/solrj_collection/leader_elect/shard2/election/99618109152231432-core_node3-n_0000000000
[junit4] 2> 2 /solr/overseer_elect/election/99618109152231448-127.0.0.1:35226__he%2Fqw-n_0000000006
[junit4] 2> 2 /solr/collections/solrj_collection2/leader_elect/shard1/election/99618109152231445-core_node4-n_0000000002
[junit4] 2> 2 /solr/overseer_elect/election/99618109152231428-127.0.0.1:34228__he%2Fqw-n_0000000000
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 12 /solr/collections
[junit4] 2> 7 /solr/live_nodes
[junit4] 2> 3 /solr/overseer/queue
[junit4] 2> 3 /solr/overseer/queue-work
[junit4] 2> 3 /solr/overseer/collection-queue-work
[junit4] 2>
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=SharedFSAutoReplicaFailoverTest -Dtests.method=test -Dtests.seed=9B90977827D05434 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/test-data/enwiki.random.lines.txt -Dtests.locale=th-TH-u-nu-thai-x-lvariant-TH -Dtests.timezone=Pacific/Wallis -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] FAILURE 194s J1 | SharedFSAutoReplicaFailoverTest.test <<<
[junit4] > Throwable #1: java.lang.AssertionError
[junit4] > at __randomizedtesting.SeedInfo.seed([9B90977827D05434:13C4A8A2892C39CC]:0)
[junit4] > at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.testBasics(SharedFSAutoReplicaFailoverTest.java:307)
[junit4] > at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test(SharedFSAutoReplicaFailoverTest.java:136)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 946148 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9B90977827D05434]-worker) [ ] o.a.h.h.s.d.DirectoryScanner DirectoryScanner: shutdown has been called
[junit4] 2> 946159 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9B90977827D05434]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
[junit4] 2> 946261 WARN (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_9B90977827D05434-001/tempDir-001/hdfsBaseDir/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_9B90977827D05434-001/tempDir-001/hdfsBaseDir/data/data4/]] heartbeating to localhost/127.0.0.1:44381) [ ] o.a.h.h.s.d.IncrementalBlockReportManager IncrementalBlockReportManager interrupted
[junit4] 2> 946261 WARN (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_9B90977827D05434-001/tempDir-001/hdfsBaseDir/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_9B90977827D05434-001/tempDir-001/hdfsBaseDir/data/data4/]] heartbeating to localhost/127.0.0.1:44381) [ ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-1675768844-10.41.0.5-1520051703658 (Datanode Uuid 6a562b5d-a072-43a4-97e3-4a80b6c44971) service to localhost/127.0.0.1:44381
[junit4] 2> 946263 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9B90977827D05434]-worker) [ ] o.a.h.h.s.d.DirectoryScanner DirectoryScanner: shutdown has been called
[junit4] 2> 946270 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9B90977827D05434]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
[junit4] 2> 946371 WARN (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_9B90977827D05434-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_9B90977827D05434-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to localhost/127.0.0.1:44381) [ ] o.a.h.h.s.d.IncrementalBlockReportManager IncrementalBlockReportManager interrupted
[junit4] 2> 946371 WARN (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_9B90977827D05434-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_9B90977827D05434-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to localhost/127.0.0.1:44381) [ ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-1675768844-10.41.0.5-1520051703658 (Datanode Uuid bab998d6-c809-4db4-8e14-6e9b79b0a53e) service to localhost/127.0.0.1:44381
[junit4] 2> 946386 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9B90977827D05434]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
[junit4] 2> 946490 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9B90977827D05434]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_9B90977827D05434-001
[junit4] 2> Mar 03, 2018 4:38:28 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 34 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): {id=PostingsFormat(name=LuceneFixedGap), text=BlockTreeOrds(blocksize=128)}, docValues:{_version_=DocValuesFormat(name=Asserting)}, maxPointsInLeafNode=258, maxMBSortInHeap=5.347407300784594, sim=RandomSimilarity(queryNorm=false,coord=no): {}, locale=th-TH-u-nu-thai-x-lvariant-TH, timezone=Pacific/Wallis
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=6,free=202325336,total=532152320
[junit4] 2> NOTE: All tests run in this JVM: [TestSolrCloudSnapshots, MoreLikeThisHandlerTest, TestRandomFlRTGCloud, TestSolrCloudWithSecureImpersonation, TestFastOutputStream, BasicFunctionalityTest, TestReloadAndDeleteDocs, IndexBasedSpellCheckerTest, FileUtilsTest, TestSizeLimitedDistributedMap, TestCoreContainer, TestSearchPerf, ChaosMonkeySafeLeaderTest, TestBlendedInfixSuggestions, TestPointFields, CollectionsAPISolrJTest, TestMinMaxOnMultiValuedField, TestManagedSchema, TestMacros, DocValuesTest, BadCopyFieldTest, MissingSegmentRecoveryTest, SolrIndexSplitterTest, SharedFSAutoReplicaFailoverTest]
[junit4] Completed [147/714 (2!)] on J1 in 214.36s, 1 test, 1 failure <<< FAILURES!
[...truncated 48150 lines...]
[JENKINS] Lucene-Solr-NightlyTests-6.6 - Build # 32 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.6/32/
3 tests failed.
FAILED: org.apache.solr.cloud.ShardSplitTest.testSplitWithChaosMonkey
Error Message:
There are still nodes recoverying - waited for 330 seconds
Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 330 seconds
at __randomizedtesting.SeedInfo.seed([34BB004ACFA3C0F3:BF9CD39B8EA56B77]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:187)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:144)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:139)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:856)
at org.apache.solr.cloud.ShardSplitTest.testSplitWithChaosMonkey(ShardSplitTest.java:437)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_dv
Error Message:
java.lang.RuntimeException: Error from server at http://127.0.0.1:33720/solr/test_col: Async exception during distributed update: Error from server at http://127.0.0.1:43289/solr/test_col_shard1_replica2: Server Error request: http://127.0.0.1:43289/solr/test_col_shard1_replica2/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A33720%2Fsolr%2Ftest_col_shard3_replica2%2F&wt=javabin&version=2 Remote error message: Failed synchronous update on shard StdNode: http://127.0.0.1:45221/solr/test_col_shard1_replica1/ update: org.apache.solr.client.solrj.request.UpdateRequest@772b5bf6
Stack Trace:
java.util.concurrent.ExecutionException: java.lang.RuntimeException: Error from server at http://127.0.0.1:33720/solr/test_col: Async exception during distributed update: Error from server at http://127.0.0.1:43289/solr/test_col_shard1_replica2: Server Error
request: http://127.0.0.1:43289/solr/test_col_shard1_replica2/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A33720%2Fsolr%2Ftest_col_shard3_replica2%2F&wt=javabin&version=2
Remote error message: Failed synchronous update on shard StdNode: http://127.0.0.1:45221/solr/test_col_shard1_replica1/ update: org.apache.solr.client.solrj.request.UpdateRequest@772b5bf6
at __randomizedtesting.SeedInfo.seed([34BB004ACFA3C0F3:2AF620C45FEFAE2]:0)
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.checkField(TestStressCloudBlindAtomicUpdates.java:281)
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_dv(TestStressCloudBlindAtomicUpdates.java:193)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: Error from server at http://127.0.0.1:33720/solr/test_col: Async exception during distributed update: Error from server at http://127.0.0.1:43289/solr/test_col_shard1_replica2: Server Error
request: http://127.0.0.1:43289/solr/test_col_shard1_replica2/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A33720%2Fsolr%2Ftest_col_shard3_replica2%2F&wt=javabin&version=2
Remote error message: Failed synchronous update on shard StdNode: http://127.0.0.1:45221/solr/test_col_shard1_replica1/ update: org.apache.solr.client.solrj.request.UpdateRequest@772b5bf6
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.run(TestStressCloudBlindAtomicUpdates.java:409)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:33720/solr/test_col: Async exception during distributed update: Error from server at http://127.0.0.1:43289/solr/test_col_shard1_replica2: Server Error
request: http://127.0.0.1:43289/solr/test_col_shard1_replica2/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A33720%2Fsolr%2Ftest_col_shard3_replica2%2F&wt=javabin&version=2
Remote error message: Failed synchronous update on shard StdNode: http://127.0.0.1:45221/solr/test_col_shard1_replica1/ update: org.apache.solr.client.solrj.request.UpdateRequest@772b5bf6
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:612)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:177)
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.doRandomAtomicUpdate(TestStressCloudBlindAtomicUpdates.java:368)
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.run(TestStressCloudBlindAtomicUpdates.java:397)
... 6 more
FAILED: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test
Error Message:
expected:<2> but was:<3>
Stack Trace:
java.lang.AssertionError: expected:<2> but was:<3>
at __randomizedtesting.SeedInfo.seed([34BB004ACFA3C0F3:BCEF3F90615FAD0B]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.assertSliceAndReplicaCount(SharedFSAutoReplicaFailoverTest.java:419)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.testBasics(SharedFSAutoReplicaFailoverTest.java:264)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test(SharedFSAutoReplicaFailoverTest.java:136)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12973 lines...]
[junit4] Suite: org.apache.solr.cloud.ShardSplitTest
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_34BB004ACFA3C0F3-001/init-core-data-001
[junit4] 2> 2751978 WARN (SUITE-ShardSplitTest-seed#[34BB004ACFA3C0F3]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
[junit4] 2> 2751979 INFO (SUITE-ShardSplitTest-seed#[34BB004ACFA3C0F3]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields
[junit4] 2> 2751980 INFO (SUITE-ShardSplitTest-seed#[34BB004ACFA3C0F3]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
[junit4] 2> 2751980 INFO (SUITE-ShardSplitTest-seed#[34BB004ACFA3C0F3]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 2> 2751995 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 2751995 INFO (Thread-3803) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2751995 INFO (Thread-3803) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 2752009 ERROR (Thread-3803) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2752095 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.c.ZkTestServer start zk server on port:33069
[junit4] 2> 2752161 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 2752181 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 2752182 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 2752183 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 2752193 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 2752194 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 2752195 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 2752217 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 2752219 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 2752237 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 2752238 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 2752684 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_34BB004ACFA3C0F3-001/control-001/cores/collection1
[junit4] 2> 2752685 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 2752686 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@188c2521{/,null,AVAILABLE}
[junit4] 2> 2752726 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@5cf6f4e6{HTTP/1.1,[http/1.1]}{127.0.0.1:36075}
[junit4] 2> 2752727 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.e.j.s.Server Started @2764404ms
[junit4] 2> 2752727 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_34BB004ACFA3C0F3-001/tempDir-001/control/data, hostContext=/, hostPort=36075, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_34BB004ACFA3C0F3-001/control-001/cores}
[junit4] 2> 2752727 ERROR (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2752727 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.6.3
[junit4] 2> 2752727 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2752727 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2752727 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-03-02T03:22:19.408Z
[junit4] 2> 2752751 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 2752751 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_34BB004ACFA3C0F3-001/control-001/solr.xml
[junit4] 2> 2752802 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 2752825 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33069/solr
[junit4] 2> 2753037 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [n:127.0.0.1:36075_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2753038 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [n:127.0.0.1:36075_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:36075_
[junit4] 2> 2753053 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [n:127.0.0.1:36075_ ] o.a.s.c.Overseer Overseer (id=99612160077135876-127.0.0.1:36075_-n_0000000000) starting
[junit4] 2> 2753110 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [n:127.0.0.1:36075_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36075_
[junit4] 2> 2753125 INFO (zkCallback-2553-thread-1-processing-n:127.0.0.1:36075_) [n:127.0.0.1:36075_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2753387 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [n:127.0.0.1:36075_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_34BB004ACFA3C0F3-001/control-001/cores
[junit4] 2> 2753387 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [n:127.0.0.1:36075_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 2753388 INFO (coreLoadExecutor-5297-thread-1-processing-n:127.0.0.1:36075_) [n:127.0.0.1:36075_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 2753446 INFO (OverseerStateUpdate-99612160077135876-127.0.0.1:36075_-n_0000000000) [n:127.0.0.1:36075_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 2754570 WARN (coreLoadExecutor-5297-thread-1-processing-n:127.0.0.1:36075_) [n:127.0.0.1:36075_ c:control_collection x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 2754571 INFO (coreLoadExecutor-5297-thread-1-processing-n:127.0.0.1:36075_) [n:127.0.0.1:36075_ c:control_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.3
[junit4] 2> 2754644 INFO (coreLoadExecutor-5297-thread-1-processing-n:127.0.0.1:36075_) [n:127.0.0.1:36075_ c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 2755099 INFO (coreLoadExecutor-5297-thread-1-processing-n:127.0.0.1:36075_) [n:127.0.0.1:36075_ c:control_collection x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 2755159 INFO (coreLoadExecutor-5297-thread-1-processing-n:127.0.0.1:36075_) [n:127.0.0.1:36075_ c:control_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection, trusted=true
[junit4] 2> 2755159 INFO (coreLoadExecutor-5297-thread-1-processing-n:127.0.0.1:36075_) [n:127.0.0.1:36075_ c:control_collection x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2755159 INFO (coreLoadExecutor-5297-thread-1-processing-n:127.0.0.1:36075_) [n:127.0.0.1:36075_ c:control_collection x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_34BB004ACFA3C0F3-001/control-001/cores/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_34BB004ACFA3C0F3-001/control-001/cores/collection1/data/]
[junit4] 2> 2755160 INFO (coreLoadExecutor-5297-thread-1-processing-n:127.0.0.1:36075_) [n:127.0.0.1:36075_ c:control_collection x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@608cc2ff
[junit4] 2> 2755174 INFO (coreLoadExecutor-5297-thread-1-processing-n:127.0.0.1:36075_) [n:127.0.0.1:36075_ c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@7b154237
[junit4] 2> 2755282 WARN (coreLoadExecutor-5297-thread-1-processing-n:127.0.0.1:36075_) [n:127.0.0.1:36075_ c:control_collection x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 2755497 INFO (coreLoadExecutor-5297-thread-1-processing-n:127.0.0.1:36075_) [n:127.0.0.1:36075_ c:control_collection x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2755497 INFO (coreLoadExecutor-5297-thread-1-processing-n:127.0.0.1:36075_) [n:127.0.0.1:36075_ c:control_collection x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2755511 INFO (coreLoadExecutor-5297-thread-1-processing-n:127.0.0.1:36075_) [n:127.0.0.1:36075_ c:control_collection x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2755511 INFO (coreLoadExecutor-5297-thread-1-processing-n:127.0.0.1:36075_) [n:127.0.0.1:36075_ c:control_collection x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2755511 INFO (coreLoadExecutor-5297-thread-1-processing-n:127.0.0.1:36075_) [n:127.0.0.1:36075_ c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=38, maxMergeAtOnceExplicit=22, maxMergedSegmentMB=1.0341796875, floorSegmentMB=0.845703125, forceMergeDeletesPctAllowed=14.694259749291467, segmentsPerTier=50.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6735098025561761
[junit4] 2> 2755531 INFO (coreLoadExecutor-5297-thread-1-processing-n:127.0.0.1:36075_) [n:127.0.0.1:36075_ c:control_collection x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1bc9afc8[collection1] main]
[junit4] 2> 2755532 INFO (coreLoadExecutor-5297-thread-1-processing-n:127.0.0.1:36075_) [n:127.0.0.1:36075_ c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2755533 INFO (coreLoadExecutor-5297-thread-1-processing-n:127.0.0.1:36075_) [n:127.0.0.1:36075_ c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2755533 INFO (coreLoadExecutor-5297-thread-1-processing-n:127.0.0.1:36075_) [n:127.0.0.1:36075_ c:control_collection x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2755534 INFO (searcherExecutor-5298-thread-1-processing-n:127.0.0.1:36075_ x:collection1 c:control_collection) [n:127.0.0.1:36075_ c:control_collection x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1bc9afc8[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2755547 INFO (coreLoadExecutor-5297-thread-1-processing-n:127.0.0.1:36075_) [n:127.0.0.1:36075_ c:control_collection x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1593794564957667328
[junit4] 2> 2755585 INFO (coreZkRegister-5290-thread-1-processing-n:127.0.0.1:36075_ x:collection1 c:control_collection) [n:127.0.0.1:36075_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2755586 INFO (coreZkRegister-5290-thread-1-processing-n:127.0.0.1:36075_ x:collection1 c:control_collection) [n:127.0.0.1:36075_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2755586 INFO (coreZkRegister-5290-thread-1-processing-n:127.0.0.1:36075_ x:collection1 c:control_collection) [n:127.0.0.1:36075_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:36075/collection1/
[junit4] 2> 2755586 INFO (coreZkRegister-5290-thread-1-processing-n:127.0.0.1:36075_ x:collection1 c:control_collection) [n:127.0.0.1:36075_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 2755586 INFO (coreZkRegister-5290-thread-1-processing-n:127.0.0.1:36075_ x:collection1 c:control_collection) [n:127.0.0.1:36075_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:36075/collection1/ has no replicas
[junit4] 2> 2755586 INFO (coreZkRegister-5290-thread-1-processing-n:127.0.0.1:36075_ x:collection1 c:control_collection) [n:127.0.0.1:36075_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 2755604 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2755606 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33069/solr ready
[junit4] 2> 2755606 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 2755606 INFO (coreZkRegister-5290-thread-1-processing-n:127.0.0.1:36075_ x:collection1 c:control_collection) [n:127.0.0.1:36075_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:36075/collection1/ shard1
[junit4] 2> 2755606 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
[junit4] 2> 2755768 INFO (coreZkRegister-5290-thread-1-processing-n:127.0.0.1:36075_ x:collection1 c:control_collection) [n:127.0.0.1:36075_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2756138 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_34BB004ACFA3C0F3-001/shard-1-001/cores/collection1
[junit4] 2> 2756139 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_34BB004ACFA3C0F3-001/shard-1-001
[junit4] 2> 2756139 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 2756154 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@25789070{/,null,AVAILABLE}
[junit4] 2> 2756155 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@784272ce{HTTP/1.1,[http/1.1]}{127.0.0.1:34046}
[junit4] 2> 2756155 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.e.j.s.Server Started @2767833ms
[junit4] 2> 2756155 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_34BB004ACFA3C0F3-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=34046, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_34BB004ACFA3C0F3-001/shard-1-001/cores}
[junit4] 2> 2756156 ERROR (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2756156 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.6.3
[junit4] 2> 2756174 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2756174 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2756174 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-03-02T03:22:22.855Z
[junit4] 2> 2756199 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 2756199 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_34BB004ACFA3C0F3-001/shard-1-001/solr.xml
[junit4] 2> 2756246 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 2756247 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33069/solr
[junit4] 2> 2756275 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [n:127.0.0.1:34046_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2756278 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [n:127.0.0.1:34046_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2756293 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [n:127.0.0.1:34046_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34046_
[junit4] 2> 2756307 INFO (zkCallback-2557-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2756307 INFO (zkCallback-2563-thread-1-processing-n:127.0.0.1:34046_) [n:127.0.0.1:34046_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2756308 INFO (zkCallback-2553-thread-2-processing-n:127.0.0.1:36075_) [n:127.0.0.1:36075_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2756503 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [n:127.0.0.1:34046_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_34BB004ACFA3C0F3-001/shard-1-001/cores
[junit4] 2> 2756503 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [n:127.0.0.1:34046_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 2756515 INFO (coreLoadExecutor-5308-thread-1-processing-n:127.0.0.1:34046_) [n:127.0.0.1:34046_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 2756517 INFO (OverseerStateUpdate-99612160077135876-127.0.0.1:36075_-n_0000000000) [n:127.0.0.1:36075_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 2756652 INFO (zkCallback-2563-thread-1-processing-n:127.0.0.1:34046_) [n:127.0.0.1:34046_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2757569 WARN (coreLoadExecutor-5308-thread-1-processing-n:127.0.0.1:34046_) [n:127.0.0.1:34046_ c:collection1 x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 2757569 INFO (coreLoadExecutor-5308-thread-1-processing-n:127.0.0.1:34046_) [n:127.0.0.1:34046_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.3
[junit4] 2> 2757614 INFO (coreLoadExecutor-5308-thread-1-processing-n:127.0.0.1:34046_) [n:127.0.0.1:34046_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 2757955 INFO (coreLoadExecutor-5308-thread-1-processing-n:127.0.0.1:34046_) [n:127.0.0.1:34046_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 2758038 INFO (coreLoadExecutor-5308-thread-1-processing-n:127.0.0.1:34046_) [n:127.0.0.1:34046_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1, trusted=true
[junit4] 2> 2758039 INFO (coreLoadExecutor-5308-thread-1-processing-n:127.0.0.1:34046_) [n:127.0.0.1:34046_ c:collection1 x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2758039 INFO (coreLoadExecutor-5308-thread-1-processing-n:127.0.0.1:34046_) [n:127.0.0.1:34046_ c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_34BB004ACFA3C0F3-001/shard-1-001/cores/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_34BB004ACFA3C0F3-001/shard-1-001/cores/collection1/data/]
[junit4] 2> 2758039 INFO (coreLoadExecutor-5308-thread-1-processing-n:127.0.0.1:34046_) [n:127.0.0.1:34046_ c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@608cc2ff
[junit4] 2> 2758041 INFO (coreLoadExecutor-5308-thread-1-processing-n:127.0.0.1:34046_) [n:127.0.0.1:34046_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@3bb647df
[junit4] 2> 2758083 WARN (coreLoadExecutor-5308-thread-1-processing-n:127.0.0.1:34046_) [n:127.0.0.1:34046_ c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 2758262 INFO (coreLoadExecutor-5308-thread-1-processing-n:127.0.0.1:34046_) [n:127.0.0.1:34046_ c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2758262 INFO (coreLoadExecutor-5308-thread-1-processing-n:127.0.0.1:34046_) [n:127.0.0.1:34046_ c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2758262 INFO (coreLoadExecutor-5308-thread-1-processing-n:127.0.0.1:34046_) [n:127.0.0.1:34046_ c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2758263 INFO (coreLoadExecutor-5308-thread-1-processing-n:127.0.0.1:34046_) [n:127.0.0.1:34046_ c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2758264 INFO (coreLoadExecutor-5308-thread-1-processing-n:127.0.0.1:34046_) [n:127.0.0.1:34046_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=38, maxMergeAtOnceExplicit=22, maxMergedSegmentMB=1.0341796875, floorSegmentMB=0.845703125, forceMergeDeletesPctAllowed=14.694259749291467, segmentsPerTier=50.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6735098025561761
[junit4] 2> 2758264 INFO (coreLoadExecutor-5308-thread-1-processing-n:127.0.0.1:34046_) [n:127.0.0.1:34046_ c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@56379b9f[collection1] main]
[junit4] 2> 2758278 INFO (coreLoadExecutor-5308-thread-1-processing-n:127.0.0.1:34046_) [n:127.0.0.1:34046_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2758278 INFO (coreLoadExecutor-5308-thread-1-processing-n:127.0.0.1:34046_) [n:127.0.0.1:34046_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2758278 INFO (coreLoadExecutor-5308-thread-1-processing-n:127.0.0.1:34046_) [n:127.0.0.1:34046_ c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2758280 INFO (searcherExecutor-5309-thread-1-processing-n:127.0.0.1:34046_ x:collection1 c:collection1) [n:127.0.0.1:34046_ c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@56379b9f[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2758280 INFO (coreLoadExecutor-5308-thread-1-processing-n:127.0.0.1:34046_) [n:127.0.0.1:34046_ c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1593794567823425536
[junit4] 2> 2758300 INFO (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:34046_ x:collection1 c:collection1) [n:127.0.0.1:34046_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2758300 INFO (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:34046_ x:collection1 c:collection1) [n:127.0.0.1:34046_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2758300 INFO (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:34046_ x:collection1 c:collection1) [n:127.0.0.1:34046_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:34046/collection1/
[junit4] 2> 2758300 INFO (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:34046_ x:collection1 c:collection1) [n:127.0.0.1:34046_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 2758300 INFO (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:34046_ x:collection1 c:collection1) [n:127.0.0.1:34046_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:34046/collection1/ has no replicas
[junit4] 2> 2758300 INFO (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:34046_ x:collection1 c:collection1) [n:127.0.0.1:34046_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 2758311 INFO (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:34046_ x:collection1 c:collection1) [n:127.0.0.1:34046_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:34046/collection1/ shard2
[junit4] 2> 2758413 INFO (zkCallback-2563-thread-1-processing-n:127.0.0.1:34046_) [n:127.0.0.1:34046_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2758463 INFO (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:34046_ x:collection1 c:collection1) [n:127.0.0.1:34046_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2758579 INFO (zkCallback-2563-thread-1-processing-n:127.0.0.1:34046_) [n:127.0.0.1:34046_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2758910 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_34BB004ACFA3C0F3-001/shard-2-001/cores/collection1
[junit4] 2> 2758910 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_34BB004ACFA3C0F3-001/shard-2-001
[junit4] 2> 2758927 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 2758928 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5439a048{/,null,AVAILABLE}
[junit4] 2> 2758929 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@35d6ab91{HTTP/1.1,[http/1.1]}{127.0.0.1:37622}
[junit4] 2> 2758929 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.e.j.s.Server Started @2770607ms
[junit4] 2> 2758929 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_34BB004ACFA3C0F3-001/tempDir-001/jetty2, solrconfig=solrconfig.xml, hostContext=/, hostPort=37622, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_34BB004ACFA3C0F3-001/shard-2-001/cores}
[junit4] 2> 2758930 ERROR (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2758930 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.6.3
[junit4] 2> 2758930 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2758930 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2758930 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-03-02T03:22:25.611Z
[junit4] 2> 2758949 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 2758949 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_34BB004ACFA3C0F3-001/shard-2-001/solr.xml
[junit4] 2> 2758955 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 2758955 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33069/solr
[junit4] 2> 2759004 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [n:127.0.0.1:37622_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2759006 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [n:127.0.0.1:37622_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2759008 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [n:127.0.0.1:37622_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37622_
[junit4] 2> 2759053 INFO (zkCallback-2563-thread-1-processing-n:127.0.0.1:34046_) [n:127.0.0.1:34046_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2759094 INFO (zkCallback-2569-thread-1-processing-n:127.0.0.1:37622_) [n:127.0.0.1:37622_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2759098 INFO (zkCallback-2553-thread-2-processing-n:127.0.0.1:36075_) [n:127.0.0.1:36075_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2759106 INFO (zkCallback-2557-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2759263 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [n:127.0.0.1:37622_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_34BB004ACFA3C0F3-001/shard-2-001/cores
[junit4] 2> 2759263 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [n:127.0.0.1:37622_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 2759305 INFO (coreLoadExecutor-5319-thread-1-processing-n:127.0.0.1:37622_) [n:127.0.0.1:37622_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 2759330 INFO (OverseerStateUpdate-99612160077135876-127.0.0.1:36075_-n_0000000000) [n:127.0.0.1:36075_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 2759434 INFO (zkCallback-2569-thread-1-processing-n:127.0.0.1:37622_) [n:127.0.0.1:37622_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2759435 INFO (zkCallback-2563-thread-1-processing-n:127.0.0.1:34046_) [n:127.0.0.1:34046_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2760412 WARN (coreLoadExecutor-5319-thread-1-processing-n:127.0.0.1:37622_) [n:127.0.0.1:37622_ c:collection1 x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 2760413 INFO (coreLoadExecutor-5319-thread-1-processing-n:127.0.0.1:37622_) [n:127.0.0.1:37622_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.3
[junit4] 2> 2760480 INFO (coreLoadExecutor-5319-thread-1-processing-n:127.0.0.1:37622_) [n:127.0.0.1:37622_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 2760807 INFO (coreLoadExecutor-5319-thread-1-processing-n:127.0.0.1:37622_) [n:127.0.0.1:37622_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 2760918 INFO (coreLoadExecutor-5319-thread-1-processing-n:127.0.0.1:37622_) [n:127.0.0.1:37622_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1, trusted=true
[junit4] 2> 2760919 INFO (coreLoadExecutor-5319-thread-1-processing-n:127.0.0.1:37622_) [n:127.0.0.1:37622_ c:collection1 x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2760919 INFO (coreLoadExecutor-5319-thread-1-processing-n:127.0.0.1:37622_) [n:127.0.0.1:37622_ c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_34BB004ACFA3C0F3-001/shard-2-001/cores/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_34BB004ACFA3C0F3-001/shard-2-001/cores/collection1/data/]
[junit4] 2> 2760919 INFO (coreLoadExecutor-5319-thread-1-processing-n:127.0.0.1:37622_) [n:127.0.0.1:37622_ c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@608cc2ff
[junit4] 2> 2760921 INFO (coreLoadExecutor-5319-thread-1-processing-n:127.0.0.1:37622_) [n:127.0.0.1:37622_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@527c86fb
[junit4] 2> 2761009 WARN (coreLoadExecutor-5319-thread-1-processing-n:127.0.0.1:37622_) [n:127.0.0.1:37622_ c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 2761185 INFO (coreLoadExecutor-5319-thread-1-processing-n:127.0.0.1:37622_) [n:127.0.0.1:37622_ c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2761185 INFO (coreLoadExecutor-5319-thread-1-processing-n:127.0.0.1:37622_) [n:127.0.0.1:37622_ c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2761186 INFO (coreLoadExecutor-5319-thread-1-processing-n:127.0.0.1:37622_) [n:127.0.0.1:37622_ c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2761186 INFO (coreLoadExecutor-5319-thread-1-processing-n:127.0.0.1:37622_) [n:127.0.0.1:37622_ c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2761199 INFO (coreLoadExecutor-5319-thread-1-processing-n:127.0.0.1:37622_) [n:127.0.0.1:37622_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=38, maxMergeAtOnceExplicit=22, maxMergedSegmentMB=1.0341796875, floorSegmentMB=0.845703125, forceMergeDeletesPctAllowed=14.694259749291467, segmentsPerTier=50.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6735098025561761
[junit4] 2> 2761200 INFO (coreLoadExecutor-5319-thread-1-processing-n:127.0.0.1:37622_) [n:127.0.0.1:37622_ c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@40f3827a[collection1] main]
[junit4] 2> 2761201 INFO (coreLoadExecutor-5319-thread-1-processing-n:127.0.0.1:37622_) [n:127.0.0.1:37622_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2761227 INFO (coreLoadExecutor-5319-thread-1-processing-n:127.0.0.1:37622_) [n:127.0.0.1:37622_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2761228 INFO (coreLoadExecutor-5319-thread-1-processing-n:127.0.0.1:37622_) [n:127.0.0.1:37622_ c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2761244 INFO (searcherExecutor-5320-thread-1-processing-n:127.0.0.1:37622_ x:collection1 c:collection1) [n:127.0.0.1:37622_ c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@40f3827a[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2761244 INFO (coreLoadExecutor-5319-thread-1-processing-n:127.0.0.1:37622_) [n:127.0.0.1:37622_ c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1593794570931404800
[junit4] 2> 2761276 INFO (coreZkRegister-5314-thread-1-processing-n:127.0.0.1:37622_ x:collection1 c:collection1) [n:127.0.0.1:37622_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2761276 INFO (coreZkRegister-5314-thread-1-processing-n:127.0.0.1:37622_ x:collection1 c:collection1) [n:127.0.0.1:37622_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2761276 INFO (coreZkRegister-5314-thread-1-processing-n:127.0.0.1:37622_ x:collection1 c:collection1) [n:127.0.0.1:37622_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:37622/collection1/
[junit4] 2> 2761277 INFO (coreZkRegister-5314-thread-1-processing-n:127.0.0.1:37622_ x:collection1 c:collection1) [n:127.0.0.1:37622_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 2761277 INFO (coreZkRegister-5314-thread-1-processing-n:127.0.0.1:37622_ x:collection1 c:collection1) [n:127.0.0.1:37622_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:37622/collection1/ has no replicas
[junit4] 2> 2761277 INFO (coreZkRegister-5314-thread-1-processing-n:127.0.0.1:37622_ x:collection1 c:collection1) [n:127.0.0.1:37622_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 2761290 INFO (coreZkRegister-5314-thread-1-processing-n:127.0.0.1:37622_ x:collection1 c:collection1) [n:127.0.0.1:37622_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:37622/collection1/ shard1
[junit4] 2> 2761398 INFO (zkCallback-2569-thread-1-processing-n:127.0.0.1:37622_) [n:127.0.0.1:37622_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2761398 INFO (zkCallback-2563-thread-1-processing-n:127.0.0.1:34046_) [n:127.0.0.1:34046_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2761409 INFO (coreZkRegister-5314-thread-1-processing-n:127.0.0.1:37622_ x:collection1 c:collection1) [n:127.0.0.1:37622_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2761520 INFO (zkCallback-2569-thread-1-processing-n:127.0.0.1:37622_) [n:127.0.0.1:37622_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2761520 INFO (zkCallback-2563-thread-1-processing-n:127.0.0.1:34046_) [n:127.0.0.1:34046_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2761784 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_34BB004ACFA3C0F3-001/shard-3-001/cores/collection1
[junit4] 2> 2761785 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_34BB004ACFA3C0F3-001/shard-3-001
[junit4] 2> 2761785 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 2761787 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@32dd3317{/,null,AVAILABLE}
[junit4] 2> 2761799 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@12513c83{HTTP/1.1,[http/1.1]}{127.0.0.1:36591}
[junit4] 2> 2761799 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.e.j.s.Server Started @2773477ms
[junit4] 2> 2761799 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_34BB004ACFA3C0F3-001/tempDir-001/jetty3, solrconfig=solrconfig.xml, hostContext=/, hostPort=36591, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_34BB004ACFA3C0F3-001/shard-3-001/cores}
[junit4] 2> 2761800 ERROR (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2761800 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.6.3
[junit4] 2> 2761801 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2761801 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2761801 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-03-02T03:22:28.482Z
[junit4] 2> 2761805 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 2761805 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_34BB004ACFA3C0F3-001/shard-3-001/solr.xml
[junit4] 2> 2761823 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 2761824 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33069/solr
[junit4] 2> 2761852 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [n:127.0.0.1:36591_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 2761855 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [n:127.0.0.1:36591_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2761869 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [n:127.0.0.1:36591_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36591_
[junit4] 2> 2761870 INFO (zkCallback-2557-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2761870 INFO (zkCallback-2569-thread-1-processing-n:127.0.0.1:37622_) [n:127.0.0.1:37622_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2761871 INFO (zkCallback-2553-thread-1-processing-n:127.0.0.1:36075_) [n:127.0.0.1:36075_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2761874 INFO (zkCallback-2575-thread-1-processing-n:127.0.0.1:36591_) [n:127.0.0.1:36591_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2761881 INFO (zkCallback-2563-thread-1-processing-n:127.0.0.1:34046_) [n:127.0.0.1:34046_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2762115 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [n:127.0.0.1:36591_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_34BB004ACFA3C0F3-001/shard-3-001/cores
[junit4] 2> 2762115 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[34BB004ACFA3C0F3]) [n:127.0.0.1:36591_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 2762151 INFO (coreLoadExecutor-5330-thread-1-processing-n:127.0.0.1:36591_) [n:127.0.0.1:36591_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 2762165 INFO (OverseerStateUpdate-99612160077135876-127.0.0.1:36075_-n_0000000000) [n:127.0.0.1:36075_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 2762276 INFO (zkCallback-2575-thread-1-processing-n:127.0.0.1:36591_) [n:127.0.0.1:36591_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2762276 INFO (zkCallback-2563-thread-1-processing-n:127.0.0.1:34046_) [n:127.0.0.1:34046_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2762276 INFO (zkCallback-2569-thread-1-processing-n:127.0.0.1:37622_) [n:127.0.0.1:37622_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2763197 WARN (coreLoadExecutor-5330-thread-1-processing-n:127.0.0.1:36591_) [n:127.0.0.1:36591_ c:collection1 x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 2763197 INFO (coreLoadExecutor-5330-thread-1-processing-n:127.0.0.1:36591_) [n:127.0.0.1:36591_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.3
[junit4] 2> 2763249 INFO (coreLoadExecutor-5330-thread-1-processing-n:127.0.0.1:36591_) [n:127.0.0.1:36591_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 2763506 INFO (coreLoadExecutor-5330-thread-1-processing-n:127.0.0.1:36591_) [n:127.0.0.1:36591_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 2763604 INFO (coreLoadExecutor-5330-thread-1-processing-n:127.0.0.1:36591_) [n:127.0.0.1:36591_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1, trusted=true
[junit4] 2> 2763605 INFO (coreLoadExecutor-5330-thread-1-processing-n:127.0.0.1:36591_) [n:127.0.0.1:36591_ c:collection1 x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2763605 INFO (coreLoadExecutor-5330-thread-1-processing-n:127.0.0.1:36591_) [n:127.0.0.1:36591_ c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_34BB004ACFA3C0F3-001/shard-3-001/cores/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_34BB004ACFA3C0F3-001/shard-3-001/cores/collection1/data/]
[junit4] 2> 2763605 INFO (coreLoadExecutor-5330-thread-1-processing-n:127.0.0.1:36591_) [n:127.0.0.1:36591_ c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@608cc2ff
[junit4] 2> 2763680 INFO (coreLoadExecutor-5330-thread-1-processing-n:127.0.0.1:36591_) [n:127.0.0.1:36591_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@2b8ef00c
[junit4] 2> 2763778 WARN (coreLoadExecutor-5330-thread-1-processing-n:127.0.0.1:36591_) [n:127.0.0.1:36591_ c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 2763966 INFO (coreLoadExecutor-5330-thread-1-processing-n:127.0.0.1:36591_) [n:127.0.0.1:36591_ c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2763966 INFO (coreLoadExecutor-5330-thread-1-processing-n:127.0.0.1:36591_) [n:127.0.0.1:36591_ c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2763967 INFO (coreLoadExecutor-5330-thread-1-processing-n:127.0.0.1:36591_) [n:127.0.0.1:36591_ c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2763967 INFO (coreLoadExecutor-5330-thread-1-processing-n:127.0.0.1:36591_) [n:127.0.0.1:36591_ c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2763990 INFO (coreLoadExecutor-5330-thread-1-processing-n:127.0.0.1:36591_) [n:127.0.0.1:36591_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=38, maxMergeAtOnceExplicit=22, maxMergedSegmentMB=1.0341796875, floorSegmentMB=0.845703125, forceMergeDeletesPctAllowed=14.694259749291467, segmentsPerTier=50.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6735098025561761
[junit4] 2> 2763991 INFO (coreLoadExecutor-5330-thread-1-processing-n:127.0.0.1:36591_) [n:127.0.0.1:36591_ c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@ca68a4e[collection1] main]
[junit4] 2> 2763993 INFO (coreLoadExecutor-5330-thread-1-processing-n:127.0.0.1:36591_) [n:127.0.0.1:36591_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2763994 INFO (coreLoadExecutor-5330-thread-1-processing-n:127.0.0.1:36591_) [n:127.0.0.1:36591_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2763994 INFO (coreLoadExecutor-5330-thread-1-processing-n:127.0.0.1:36591_) [n:127.0.0.1:36591_ c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2763995 INFO (searcherExecutor-5331-thread-1-processing-n:127.0.0.1:36591_ x:collection1 c:collection1) [n:127.0.0.1:36591_ c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@ca68a4e[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2763996 INFO (coreLoadExecutor-5330-thread-1-processing-n:127.0.0.1:36591_) [n:127.0.0.1:36591_ c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1593794573817085952
[junit4] 2> 2764012 INFO (coreZkRegister-5325-thread-1-processing-n:127.0.0.1:36591_ x:collection1 c:collection1) [n:127.0.0.1:36591_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.ZkController Core needs to recover:collection1
[junit4] 2> 2764012 INFO (updateExecutor-2572-thread-1-processing-n:127.0.0.1:36591_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:36591_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.DefaultSolrCoreState Running recovery
[junit4] 2> 2764012 INFO (recoveryExecutor-2573-thread-1-processing-n:127.0.0.1:36591_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:36591_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
[junit4] 2> 2764041 INFO (recoveryExecutor-2573-thread-1-processing-n:127.0.0.1:36591_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:36591_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
[junit4] 2> 2764041 INFO (recoveryExecutor-2573-thread-1-processing-n:127.0.0.1:36591_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:36591_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1]
[junit4] 2> 2764041 INFO (recoveryExecutor-2573-thread-1-processing-n:127.0.0.1:36591_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:36591_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 2764041 INFO (recoveryExecutor-2573-thread-1-processing-n:127.0.0.1:36591_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:36591_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Publishing state of core [collection1] as recovering, leader is [http://127.0.0.1:34046/collection1/] and I am [http://127.0.0.1:36591/collection1/]
[junit4] 2> 2764043 INFO (recoveryExecutor-2573-thread-1-processing-n:127.0.0.1:36591_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:36591_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:34046]; [WaitForState: action=PREPRECOVERY&core=collection1&nodeName=127.0.0.1:36591_&coreNodeName=core_node3&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
[junit4] 2> 2764079 INFO (qtp1820166628-14961) [n:127.0.0.1:34046_ ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true
[junit4] 2> 2764080 INFO (qtp1820166628-14961) [n:127.0.0.1:34046_ ] o.a.s.h.a.PrepRecoveryOp Will wait a max of 183 seconds to see collection1 (shard2 of collection1) have state: recovering
[junit4] 2> 2764080 INFO (qtp1820166628-14961) [n:127.0.0.1:34046_ ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard2, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:36591_, coreNodeName=core_node3, onlyIfActiveCheckResult=false, nodeProps: core_node3:{"core":"collection1","base_url":"http://127.0.0.1:36591","node_name":"127.0.0.1:36591_","state":"down"}
[junit4] 2> 2764180 INFO (zkCallback-2575-thread-1-processing-n:127.0.0.1:36591_) [n:127.0.0.1:36591_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2764180 INFO (zkCallback-2569-thread-1-processing-n:127.0.0.1:37622_) [n:127.0.0.1:37622_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2764180 INFO (zkCallback-2563-thread-1-processing-n:127.0.0.1:34046_) [n:127.0.0.1:34046_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2765085 INFO (qtp1820166628-14961) [n:127.0.0.1:34046_ ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard2, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:36591_, coreNodeName=core_node3, onlyIfActiveCheckResult=false, nodeProps: core_node3:{"core":"collection1","base_url":"http://127.0.0.1:36591","node_name":"127.0.0.1:36591_","state":"recovering"}
[junit4] 2> 2765085 INFO (qtp1820166628-14961) [n:127.0.0.1:34046_ ] o.a.s.h.a.PrepRecoveryOp Waited coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4] 2> 2765085 INFO (qtp1820166628-14961) [n:127.0.0.1:34046_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:36591_&onlyIfLeaderActive=true&core=collection1&coreNodeName=core_node3&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=1006
[junit4] 2> 2765597 INFO (recoveryExecutor-2573-thread-1-processing-n:127.0.0.1:36591_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:36591_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Attempting to PeerSync from [http://127.0.0.1:34046/collection1/] - recoveringAfterStartup=[true]
[junit4] 2> 2765597 INFO (recoveryExecutor-2573-thread-1-processing-n:127.0.0.1:36591_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:36591_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.PeerSync PeerSync: core=collection1 url=http://127.0.0.1:36591 START replicas=[http://127.0.0.1:34046/collection1/] nUpdates=100
[junit4] 2> 2765645 INFO (qtp1820166628-14959) [n:127.0.0.1:34046_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.IndexFingerprint IndexFingerprint millis:1.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
[junit4] 2> 2765645 INFO (qtp1820166628-14959) [n:127.0.0.1:34046_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/get params={distrib=false&qt=/get&getFingerprint=9223372036854775807&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 2765775 INFO (recoveryExecutor-2573-thread-1-processing-n:127.0.0.1:36591_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:36591_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.IndexFingerprint IndexFingerprint millis:36.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
[junit4] 2> 2765775 INFO (recoveryExecutor-2573-thread-1-processing-n:127.0.0.1:36591_ x:collection1 s:shard2 c:collection1 r:core_node3) [n
[...truncated too long message...]
ng-n:127.0.0.1:33720_solr) [n:127.0.0.1:33720_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 6881554 INFO (jetty-closer-15029-thread-3) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@4eeab602{/solr,null,UNAVAILABLE}
[junit4] 2> 6881704 WARN (zkCallback-15076-thread-3-processing-n:127.0.0.1:36461_solr) [n:127.0.0.1:36461_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 6881704 INFO (jetty-closer-15029-thread-6) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@78c1e86b{/solr,null,UNAVAILABLE}
[junit4] 2> 6882028 WARN (zkCallback-15059-thread-2-processing-n:127.0.0.1:43845_solr) [n:127.0.0.1:43845_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 6882029 INFO (jetty-closer-15029-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@3e018eaa{/solr,null,UNAVAILABLE}
[junit4] 2> 6882255 WARN (zkCallback-15083-thread-2-processing-n:127.0.0.1:41903_solr) [n:127.0.0.1:41903_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 6882255 INFO (jetty-closer-15029-thread-5) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5bf1a9ad{/solr,null,UNAVAILABLE}
[junit4] 2> 6882276 ERROR (SUITE-TestStressCloudBlindAtomicUpdates-seed#[34BB004ACFA3C0F3]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 6882277 INFO (SUITE-TestStressCloudBlindAtomicUpdates-seed#[34BB004ACFA3C0F3]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:45514 45514
[junit4] 2> 6882393 INFO (Thread-66670) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:45514 45514
[junit4] 2> 6882394 WARN (Thread-66670) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 11 /solr/aliases.json
[junit4] 2> 11 /solr/clusterprops.json
[junit4] 2> 10 /solr/configs/org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates_config-set
[junit4] 2> 10 /solr/security.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 11 /solr/clusterstate.json
[junit4] 2> 10 /solr/collections/test_col/state.json
[junit4] 2> 2 /solr/overseer_elect/election/99612404653031433-127.0.0.1:41509_solr-n_0000000000
[junit4] 2> 2 /solr/overseer_elect/election/99612404653031439-127.0.0.1:43289_solr-n_0000000003
[junit4] 2> 2 /solr/overseer_elect/election/99612404653031452-127.0.0.1:41903_solr-n_0000000006
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 11 /solr/live_nodes
[junit4] 2> 11 /solr/collections
[junit4] 2>
[junit4] 2> 6882394 INFO (SUITE-TestStressCloudBlindAtomicUpdates-seed#[34BB004ACFA3C0F3]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_34BB004ACFA3C0F3-001
[junit4] 2> Mar 02, 2018 4:31:08 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): {long_stored_idx=PostingsFormat(name=LuceneVarGapDocFreqInterval), long_dv_idx=Lucene50(blocksize=128), id=FST50, long_dv_stored_idx=Lucene50(blocksize=128)}, docValues:{long_stored_idx=DocValuesFormat(name=Lucene54), _version_=DocValuesFormat(name=Lucene54), long_dv_idx=DocValuesFormat(name=Asserting), long_dv_stored=DocValuesFormat(name=Direct), id=DocValuesFormat(name=Direct), long_dv=DocValuesFormat(name=Asserting), long_dv_stored_idx=DocValuesFormat(name=Asserting)}, maxPointsInLeafNode=1923, maxMBSortInHeap=5.189996557949406, sim=RandomSimilarity(queryNorm=false,coord=no): {}, locale=bg-BG, timezone=CTT
[junit4] 2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=1,free=130046464,total=531628032
[junit4] 2> NOTE: All tests run in this JVM: [TestGeoJSONResponseWriter, TestClusterStateMutator, ScriptEngineTest, ManagedSchemaRoundRobinCloudTest, TestBinaryResponseWriter, TestRandomCollapseQParserPlugin, TestFieldCacheSanityChecker, BufferStoreTest, TestSolrConfigHandler, SolrMetricReporterTest, BadCopyFieldTest, BJQParserTest, TestApiFramework, TestManagedResourceStorage, TestFieldCacheVsDocValues, ExitableDirectoryReaderTest, TestCloudSchemaless, TestReplicationHandlerBackup, TestExactStatsCache, LeaderElectionContextKeyTest, TestLeaderElectionWithEmptyReplica, TestZkAclsWithHadoopAuth, TestValueSourceCache, TestNumericTerms32, LeaderElectionIntegrationTest, TestStressUserVersions, TestBlendedInfixSuggestions, TestPushWriter, TestCloudJSONFacetJoinDomain, TestOmitPositions, TestPseudoReturnFields, TestFilteredDocIdSet, TestMultiWordSynonyms, WrapperMergePolicyFactoryTest, TestFunctionQuery, TestStressRecovery, TestLocalFSCloudBackupRestore, TestDelegationWithHadoopAuth, DeleteStatusTest, AliasIntegrationTest, CurrencyFieldOpenExchangeTest, TestCodecSupport, RecoveryAfterSoftCommitTest, DistributedQueryElevationComponentTest, SolrSlf4jReporterTest, TestLazyCores, DocValuesNotIndexedTest, TestComplexPhraseQParserPlugin, TestCustomStream, DistributedFacetPivotSmallAdvancedTest, TestPathTrie, PolyFieldTest, ClassificationUpdateProcessorTest, OverseerCollectionConfigSetProcessorTest, TestCloudRecovery, InfoHandlerTest, StandardRequestHandlerTest, ReturnFieldsTest, SolrCoreTest, MissingSegmentRecoveryTest, TestDynamicFieldResource, TestHighFrequencyDictionaryFactory, SecurityConfHandlerTest, TestDistributedSearch, UniqFieldsUpdateProcessorFactoryTest, TestJsonRequest, LeaderInitiatedRecoveryOnCommitTest, TestConfigSetsAPI, BasicDistributedZkTest, HdfsSyncSliceTest, AddBlockUpdateTest, ConfigSetsAPITest, OutputWriterTest, ZkNodePropsTest, BaseCdcrDistributedZkTest, TestFoldingMultitermQuery, IgnoreCommitOptimizeUpdateProcessorFactoryTest, TestJettySolrRunner, ChaosMonkeyNothingIsSafeTest, TestDocBasedVersionConstraints, SyncSliceTest, ExternalFileFieldSortTest, FileBasedSpellCheckerTest, MBeansHandlerTest, ShufflingReplicaListTransformerTest, TestReplicaProperties, HighlighterMaxOffsetTest, TestClassicSimilarityFactory, TestInitQParser, TestDocSet, TestPerFieldSimilarity, TestSchemaResource, TestInPlaceUpdatesDistrib, SignatureUpdateProcessorFactoryTest, TestPayloadCheckQParserPlugin, DeleteNodeTest, TestSolrCoreParser, TestSSLRandomization, TestBulkSchemaConcurrent, ExplicitHLLTest, TestJmxIntegration, PeerSyncTest, DistribJoinFromCollectionTest, DataDrivenBlockJoinTest, DistributedQueryComponentCustomSortTest, DisMaxRequestHandlerTest, TestMinMaxOnMultiValuedField, VersionInfoTest, TestPivotHelperCode, TestSolrCLIRunExample, TestClassNameShortening, TestReloadAndDeleteDocs, QueryEqualityTest, HdfsRecoverLeaseTest, TestPHPSerializedResponseWriter, BasicDistributedZk2Test, UnloadDistributedZkTest, OpenCloseCoreStressTest, BasicZkTest, TestReplicationHandler, ZkCLITest, TestDistributedGrouping, DistributedSpellCheckComponentTest, ZkControllerTest, TestRealTimeGet, TestJoin, TestStressVersions, DistributedTermsComponentTest, TestSolr4Spatial, QueryElevationComponentTest, TestFiltering, CurrencyFieldXmlFileTest, AnalysisAfterCoreReloadTest, TestExtendedDismaxParser, SolrRequestParserTest, SuggesterTest, TestStressLucene, TestCSVLoader, SolrCoreCheckLockOnStartupTest, TestUpdate, TestAtomicUpdateErrorCases, DirectUpdateHandlerOptimizeTest, StatelessScriptUpdateProcessorFactoryTest, TestSolrDeletionPolicy1, XsltUpdateRequestHandlerTest, CacheHeaderTest, TestQueryUtils, DirectSolrSpellCheckerTest, DocumentAnalysisRequestHandlerTest, PrimitiveFieldTypeTest, PathHierarchyTokenizerFactoryTest, FastVectorHighlighterTest, LoggingHandlerTest, SolrPluginUtilsTest, UpdateRequestProcessorFactoryTest, TestCSVResponseWriter, TestAnalyzedSuggestions, TestPartialUpdateDeduplication, PingRequestHandlerTest, BadComponentTest, TestSolrDeletionPolicy2, SampleTest, TestSearchPerf, MinimalSchemaTest, TestConfig, TestLuceneMatchVersion, TestCharFilters, TestIBSimilarityFactory, TimeZoneUtilsTest, URLClassifyProcessorTest, DateMathParserTest, TestSystemIdResolver, DateFieldTest, SpellingQueryConverterTest, RAMDirectoryFactoryTest, ClusterStateTest, TestLRUCache, TestUtils, SliceStateTest, SystemInfoHandlerTest, FileUtilsTest, DistributedMLTComponentTest, DistributedIntervalFacetingTest, CdcrRequestHandlerTest, CollectionsAPISolrJTest, DeleteInactiveReplicaTest, DeleteReplicaTest, DistribDocExpirationUpdateProcessorTest, DistributedQueueTest, DistributedVersionInfoTest, LeaderFailureAfterFreshStartTest, MigrateRouteKeyTest, RemoteQueryErrorTest, ReplicationFactorTest, RestartWhileUpdatingTest, SolrCLIZkUtilsTest, SolrCloudExampleTest, SolrXmlInZkTest, TestAuthenticationFramework, TestCloudDeleteByQuery, TestCloudPseudoReturnFields, TestClusterProperties, TestConfigSetsAPIExclusivity, TestConfigSetsAPIZkFailure, TestDistribDocBasedVersion, TestDownShardTolerantSearch, TestHdfsCloudBackupRestore, TestMiniSolrCloudClusterSSL, TestSizeLimitedDistributedMap, TestSolrCloudWithKerberosAlt, TestSolrCloudWithSecureImpersonation, TestStressCloudBlindAtomicUpdates]
[junit4] Completed [538/713 (3!)] on J0 in 398.66s, 5 tests, 1 error <<< FAILURES!
[...truncated 46891 lines...]
[JENKINS] Lucene-Solr-NightlyTests-6.6 - Build # 31 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.6/31/
7 tests failed.
FAILED: org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([62216138BDCB1501]:0)
FAILED: junit.framework.TestSuite.org.apache.lucene.spatial3d.TestGeo3DPoint
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([62216138BDCB1501]:0)
FAILED: org.apache.solr.cloud.CdcrReplicationHandlerTest.testReplicationWithBufferedUpdates
Error Message:
Timeout while trying to assert number of documents @ source_collection
Stack Trace:
java.lang.AssertionError: Timeout while trying to assert number of documents @ source_collection
at __randomizedtesting.SeedInfo.seed([E1BDE91CA304AC18:32B4B902E697308F]:0)
at org.apache.solr.cloud.BaseCdcrDistributedZkTest.assertNumDocs(BaseCdcrDistributedZkTest.java:271)
at org.apache.solr.cloud.CdcrReplicationHandlerTest.testReplicationWithBufferedUpdates(CdcrReplicationHandlerTest.java:233)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.AssertionError: expected:<90> but was:<100>
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.BaseCdcrDistributedZkTest.assertNumDocs(BaseCdcrDistributedZkTest.java:262)
... 42 more
FAILED: org.apache.solr.cloud.RestartWhileUpdatingTest.test
Error Message:
shard1 is not consistent. Got 3013 from https://127.0.0.1:41188/se_/collection1 (previous client) and got 2275 from https://127.0.0.1:46158/se_/collection1
Stack Trace:
java.lang.AssertionError: shard1 is not consistent. Got 3013 from https://127.0.0.1:41188/se_/collection1 (previous client) and got 2275 from https://127.0.0.1:46158/se_/collection1
at __randomizedtesting.SeedInfo.seed([E1BDE91CA304AC18:69E9D6C60DF8C1E0]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1271)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1250)
at org.apache.solr.cloud.RestartWhileUpdatingTest.test(RestartWhileUpdatingTest.java:155)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.hdfs.HdfsUnloadDistributedZkTest.test
Error Message:
Captured an uncaught exception in thread: Thread[id=186124, name=testExecutor-11230-thread-6, state=RUNNABLE, group=TGRP-HdfsUnloadDistributedZkTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=186124, name=testExecutor-11230-thread-6, state=RUNNABLE, group=TGRP-HdfsUnloadDistributedZkTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:42094/zmwx/hr
at __randomizedtesting.SeedInfo.seed([E1BDE91CA304AC18]:0)
at org.apache.solr.cloud.UnloadDistributedZkTest.lambda$testUnloadLotsOfCores$0(UnloadDistributedZkTest.java:412)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:42094/zmwx/hr
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:623)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at org.apache.solr.cloud.UnloadDistributedZkTest.lambda$testUnloadLotsOfCores$0(UnloadDistributedZkTest.java:410)
... 4 more
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:515)
... 8 more
FAILED: org.apache.solr.cloud.hdfs.StressHdfsTest.test
Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:33709/xt
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:33709/xt
at __randomizedtesting.SeedInfo.seed([E1BDE91CA304AC18:69E9D6C60DF8C1E0]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:623)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:447)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:388)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1383)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1134)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1073)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at org.apache.solr.cloud.hdfs.StressHdfsTest.createAndDeleteCollection(StressHdfsTest.java:220)
at org.apache.solr.cloud.hdfs.StressHdfsTest.test(StressHdfsTest.java:103)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:515)
... 51 more
FAILED: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test
Error Message:
expected:<2> but was:<4>
Stack Trace:
java.lang.AssertionError: expected:<2> but was:<4>
at __randomizedtesting.SeedInfo.seed([E1BDE91CA304AC18:69E9D6C60DF8C1E0]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.assertSliceAndReplicaCount(SharedFSAutoReplicaFailoverTest.java:419)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.testBasics(SharedFSAutoReplicaFailoverTest.java:264)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test(SharedFSAutoReplicaFailoverTest.java:136)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 10433 lines...]
[junit4] Suite: org.apache.lucene.spatial3d.TestGeo3DPoint
[junit4] 2> DFómh 16, 2017 10:39:43 A.M. com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
[junit4] 2> WARNING: Suite execution timed out: org.apache.lucene.spatial3d.TestGeo3DPoint
[junit4] 2> 1) Thread[id=9, name=JUnit4-serializer-daemon, state=TIMED_WAITING, group=main]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:50)
[junit4] 2> 2) Thread[id=14, name=TEST-TestGeo3DPoint.testRandomBig-seed#[62216138BDCB1501], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoComplexPolygon.isWithin(GeoComplexPolygon.java:326)
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoBaseMembershipShape.isWithin(GeoBaseMembershipShape.java:36)
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoBaseShape.getBounds(GeoBaseShape.java:49)
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoComplexPolygon.getBounds(GeoComplexPolygon.java:375)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.verify(TestGeo3DPoint.java:845)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.doTestRandom(TestGeo3DPoint.java:548)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig(TestGeo3DPoint.java:480)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 3) Thread[id=1, name=main, state=WAITING, group=main]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Thread.join(Thread.java:1252)
[junit4] 2> at java.lang.Thread.join(Thread.java:1326)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:608)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:457)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:244)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:355)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:13)
[junit4] 2> 4) Thread[id=13, name=SUITE-TestGeo3DPoint-seed#[62216138BDCB1501], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at java.lang.Thread.getStackTrace(Thread.java:1559)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:696)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:693)
[junit4] 2> at java.security.AccessController.doPrivileged(Native Method)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getStackTrace(ThreadLeakControl.java:693)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getThreadsWithTraces(ThreadLeakControl.java:709)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:689)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:65)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:415)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:678)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:140)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:598)
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestGeo3DPoint -Dtests.method=testRandomBig -Dtests.seed=62216138BDCB1501 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/test-data/enwiki.random.lines.txt -Dtests.locale=ga -Dtests.timezone=America/Montreal -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 7200s J1 | TestGeo3DPoint.testRandomBig <<<
[junit4] > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
[junit4] > at __randomizedtesting.SeedInfo.seed([62216138BDCB1501]:0)
[junit4] 2> DFómh 16, 2017 10:39:43 A.M. com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> DFómh 16, 2017 10:40:03 A.M. com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.lucene.spatial3d.TestGeo3DPoint:
[junit4] 2> 1) Thread[id=14, name=TEST-TestGeo3DPoint.testRandomBig-seed#[62216138BDCB1501], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at org.apache.lucene.spatial3d.geom.Plane.getSampleIntersectionPoint(Plane.java:2356)
[junit4] 2> at org.apache.lucene.spatial3d.geom.StandardXYZSolid.<init>(StandardXYZSolid.java:217)
[junit4] 2> at org.apache.lucene.spatial3d.geom.XYZSolidFactory.makeXYZSolid(XYZSolidFactory.java:64)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.verify(TestGeo3DPoint.java:846)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.doTestRandom(TestGeo3DPoint.java:548)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig(TestGeo3DPoint.java:480)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> DFómh 16, 2017 10:40:03 A.M. com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=14, name=TEST-TestGeo3DPoint.testRandomBig-seed#[62216138BDCB1501], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> DFómh 16, 2017 10:40:06 A.M. com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> SEVERE: There are still zombie threads that couldn't be terminated:
[junit4] 2> 1) Thread[id=14, name=TEST-TestGeo3DPoint.testRandomBig-seed#[62216138BDCB1501], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at org.apache.lucene.spatial3d.geom.Plane.findIntersectionBounds(Plane.java:1097)
[junit4] 2> at org.apache.lucene.spatial3d.geom.Plane.recordBounds(Plane.java:1214)
[junit4] 2> at org.apache.lucene.spatial3d.geom.XYZBounds.addIntersection(XYZBounds.java:262)
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoStandardPath$PathSegment.getBounds(GeoStandardPath.java:816)
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoStandardPath.getBounds(GeoStandardPath.java:298)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.verify(TestGeo3DPoint.java:845)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.doTestRandom(TestGeo3DPoint.java:548)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig(TestGeo3DPoint.java:480)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): {id=FSTOrd50}, docValues:{id=DocValuesFormat(name=Direct), point=DocValuesFormat(name=Memory)}, maxPointsInLeafNode=919, maxMBSortInHeap=5.819891880591646, sim=RandomSimilarity(queryNorm=false,coord=yes): {}, locale=ga, timezone=America/Montreal
[junit4] 2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=2,free=258347264,total=480772096
[junit4] 2> NOTE: All tests run in this JVM: [GeoPointTest, TestGeo3DPoint]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestGeo3DPoint -Dtests.seed=62216138BDCB1501 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/test-data/enwiki.random.lines.txt -Dtests.locale=ga -Dtests.timezone=America/Montreal -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J1 | TestGeo3DPoint (suite) <<<
[junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4] > at __randomizedtesting.SeedInfo.seed([62216138BDCB1501]:0)
[junit4] Completed [11/11 (1!)] on J1 in 7223.63s, 1 test, 2 errors <<< FAILURES!
[...truncated 1793 lines...]
[junit4] Suite: org.apache.solr.cloud.hdfs.StressHdfsTest
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_E1BDE91CA304AC18-001/init-core-data-001
[junit4] 2> 681944 WARN (SUITE-StressHdfsTest-seed#[E1BDE91CA304AC18]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=27 numCloses=27
[junit4] 2> 681944 INFO (SUITE-StressHdfsTest-seed#[E1BDE91CA304AC18]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields
[junit4] 2> 681945 INFO (SUITE-StressHdfsTest-seed#[E1BDE91CA304AC18]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
[junit4] 2> 681945 INFO (SUITE-StressHdfsTest-seed#[E1BDE91CA304AC18]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /xt/
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 682397 WARN (SUITE-StressHdfsTest-seed#[E1BDE91CA304AC18]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 682533 WARN (SUITE-StressHdfsTest-seed#[E1BDE91CA304AC18]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 682535 INFO (SUITE-StressHdfsTest-seed#[E1BDE91CA304AC18]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 682643 INFO (SUITE-StressHdfsTest-seed#[E1BDE91CA304AC18]-worker) [ ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/hdfs to ./temp/Jetty_lucene2.us.west_apache_org_45792_hdfs____d89dcp/webapp
[junit4] 2> 684195 INFO (SUITE-StressHdfsTest-seed#[E1BDE91CA304AC18]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@lucene2-us-west.apache.org:45792
[junit4] 2> 685528 WARN (SUITE-StressHdfsTest-seed#[E1BDE91CA304AC18]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 685529 INFO (SUITE-StressHdfsTest-seed#[E1BDE91CA304AC18]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 685640 INFO (SUITE-StressHdfsTest-seed#[E1BDE91CA304AC18]-worker) [ ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_42110_datanode____cto3w9/webapp
[junit4] 2> 687270 INFO (SUITE-StressHdfsTest-seed#[E1BDE91CA304AC18]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42110
[junit4] 2> 688163 WARN (SUITE-StressHdfsTest-seed#[E1BDE91CA304AC18]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 688164 INFO (SUITE-StressHdfsTest-seed#[E1BDE91CA304AC18]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 688310 INFO (SUITE-StressHdfsTest-seed#[E1BDE91CA304AC18]-worker) [ ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_41279_datanode____ne5owu/webapp
[junit4] 2> 689262 ERROR (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_E1BDE91CA304AC18-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_E1BDE91CA304AC18-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:42643) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 689734 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x1d6665dbc6a078: from storage DS-35f6830d-12df-49db-a539-fbf83a144630 node DatanodeRegistration(127.0.0.1:43242, datanodeUuid=40559efb-2fa6-4566-a879-ea9df198f2a5, infoPort=35486, infoSecurePort=0, ipcPort=43055, storageInfo=lv=-56;cid=testClusterID;nsid=1914603462;c=0), blocks: 0, hasStaleStorage: true, processing time: 87 msecs
[junit4] 2> 689754 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x1d6665dbc6a078: from storage DS-8a3641f0-6e3c-4270-847c-c5375c7ad87c node DatanodeRegistration(127.0.0.1:43242, datanodeUuid=40559efb-2fa6-4566-a879-ea9df198f2a5, infoPort=35486, infoSecurePort=0, ipcPort=43055, storageInfo=lv=-56;cid=testClusterID;nsid=1914603462;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 690222 INFO (SUITE-StressHdfsTest-seed#[E1BDE91CA304AC18]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41279
[junit4] 2> 691404 ERROR (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_E1BDE91CA304AC18-001/tempDir-001/hdfsBaseDir/data/data3/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_E1BDE91CA304AC18-001/tempDir-001/hdfsBaseDir/data/data4/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:42643) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 691536 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x1d66664b5c31d2: from storage DS-041e987e-c3e7-4367-88c8-f8e35e37db4c node DatanodeRegistration(127.0.0.1:46619, datanodeUuid=c51d34e1-bb67-4235-8678-ecfcb412335a, infoPort=34053, infoSecurePort=0, ipcPort=40925, storageInfo=lv=-56;cid=testClusterID;nsid=1914603462;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 691536 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x1d66664b5c31d2: from storage DS-02980c51-d5ca-436f-8574-3f8813f8160a node DatanodeRegistration(127.0.0.1:46619, datanodeUuid=c51d34e1-bb67-4235-8678-ecfcb412335a, infoPort=34053, infoSecurePort=0, ipcPort=40925, storageInfo=lv=-56;cid=testClusterID;nsid=1914603462;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 691769 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 691790 INFO (Thread-897) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 691790 INFO (Thread-897) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 691844 ERROR (Thread-897) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 691892 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.a.s.c.ZkTestServer start zk server on port:37605
[junit4] 2> 692049 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 692064 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 692065 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 692066 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 692103 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 692157 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 692158 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 692159 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 692172 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 692173 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 692174 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 692734 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_E1BDE91CA304AC18-001/control-001/cores/collection1
[junit4] 2> 692735 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 692769 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@348e21d0{/xt,null,AVAILABLE}
[junit4] 2> 692770 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@28369398{HTTP/1.1,[http/1.1]}{127.0.0.1:34034}
[junit4] 2> 692770 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.e.j.s.Server Started @706642ms
[junit4] 2> 692770 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://lucene2-us-west.apache.org:42643/hdfs__lucene2-us-west.apache.org_42643__home_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-6.6_checkout_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.StressHdfsTest_E1BDE91CA304AC18-001_tempDir-002_control_data, hostContext=/xt, hostPort=34034, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_E1BDE91CA304AC18-001/control-001/cores}
[junit4] 2> 692770 ERROR (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 692770 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.6.2
[junit4] 2> 692770 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 692770 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 692770 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-10-16T15:00:55.837Z
[junit4] 2> 692960 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 692961 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_E1BDE91CA304AC18-001/control-001/solr.xml
[junit4] 2> 693091 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 693127 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37605/solr
[junit4] 2> 693292 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [n:127.0.0.1:34034_xt ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 693293 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [n:127.0.0.1:34034_xt ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:34034_xt
[junit4] 2> 693293 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [n:127.0.0.1:34034_xt ] o.a.s.c.Overseer Overseer (id=98839170572484612-127.0.0.1:34034_xt-n_0000000000) starting
[junit4] 2> 693350 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [n:127.0.0.1:34034_xt ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34034_xt
[junit4] 2> 693400 INFO (zkCallback-262-thread-1-processing-n:127.0.0.1:34034_xt) [n:127.0.0.1:34034_xt ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 693844 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [n:127.0.0.1:34034_xt ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_E1BDE91CA304AC18-001/control-001/cores
[junit4] 2> 693844 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [n:127.0.0.1:34034_xt ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 693855 INFO (coreLoadExecutor-922-thread-1-processing-n:127.0.0.1:34034_xt) [n:127.0.0.1:34034_xt ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 693952 INFO (OverseerStateUpdate-98839170572484612-127.0.0.1:34034_xt-n_0000000000) [n:127.0.0.1:34034_xt ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 694976 INFO (coreLoadExecutor-922-thread-1-processing-n:127.0.0.1:34034_xt) [n:127.0.0.1:34034_xt c:control_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.2
[junit4] 2> 695064 INFO (coreLoadExecutor-922-thread-1-processing-n:127.0.0.1:34034_xt) [n:127.0.0.1:34034_xt c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 695971 INFO (coreLoadExecutor-922-thread-1-processing-n:127.0.0.1:34034_xt) [n:127.0.0.1:34034_xt c:control_collection x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 696064 INFO (coreLoadExecutor-922-thread-1-processing-n:127.0.0.1:34034_xt) [n:127.0.0.1:34034_xt c:control_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection, trusted=true
[junit4] 2> 696081 INFO (coreLoadExecutor-922-thread-1-processing-n:127.0.0.1:34034_xt) [n:127.0.0.1:34034_xt c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://lucene2-us-west.apache.org:42643/solr_hdfs_home
[junit4] 2> 696082 INFO (coreLoadExecutor-922-thread-1-processing-n:127.0.0.1:34034_xt) [n:127.0.0.1:34034_xt c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 696082 INFO (coreLoadExecutor-922-thread-1-processing-n:127.0.0.1:34034_xt) [n:127.0.0.1:34034_xt c:control_collection x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 696082 INFO (coreLoadExecutor-922-thread-1-processing-n:127.0.0.1:34034_xt) [n:127.0.0.1:34034_xt c:control_collection x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_E1BDE91CA304AC18-001/control-001/cores/collection1], dataDir=[hdfs://lucene2-us-west.apache.org:42643/solr_hdfs_home/control_collection/core_node1/data/]
[junit4] 2> 696082 INFO (coreLoadExecutor-922-thread-1-processing-n:127.0.0.1:34034_xt) [n:127.0.0.1:34034_xt c:control_collection x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@616b5c77
[junit4] 2> 696099 INFO (coreLoadExecutor-922-thread-1-processing-n:127.0.0.1:34034_xt) [n:127.0.0.1:34034_xt c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:42643/solr_hdfs_home/control_collection/core_node1/data/snapshot_metadata
[junit4] 2> 696191 INFO (coreLoadExecutor-922-thread-1-processing-n:127.0.0.1:34034_xt) [n:127.0.0.1:34034_xt c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 696191 INFO (coreLoadExecutor-922-thread-1-processing-n:127.0.0.1:34034_xt) [n:127.0.0.1:34034_xt c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 696191 INFO (coreLoadExecutor-922-thread-1-processing-n:127.0.0.1:34034_xt) [n:127.0.0.1:34034_xt c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 699786 INFO (coreLoadExecutor-922-thread-1-processing-n:127.0.0.1:34034_xt) [n:127.0.0.1:34034_xt c:control_collection x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 699844 INFO (coreLoadExecutor-922-thread-1-processing-n:127.0.0.1:34034_xt) [n:127.0.0.1:34034_xt c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:42643/solr_hdfs_home/control_collection/core_node1/data
[junit4] 2> 700135 INFO (coreLoadExecutor-922-thread-1-processing-n:127.0.0.1:34034_xt) [n:127.0.0.1:34034_xt c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:42643/solr_hdfs_home/control_collection/core_node1/data/index
[junit4] 2> 700211 INFO (coreLoadExecutor-922-thread-1-processing-n:127.0.0.1:34034_xt) [n:127.0.0.1:34034_xt c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 700211 INFO (coreLoadExecutor-922-thread-1-processing-n:127.0.0.1:34034_xt) [n:127.0.0.1:34034_xt c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 700211 INFO (coreLoadExecutor-922-thread-1-processing-n:127.0.0.1:34034_xt) [n:127.0.0.1:34034_xt c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 700355 INFO (coreLoadExecutor-922-thread-1-processing-n:127.0.0.1:34034_xt) [n:127.0.0.1:34034_xt c:control_collection x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 700355 INFO (coreLoadExecutor-922-thread-1-processing-n:127.0.0.1:34034_xt) [n:127.0.0.1:34034_xt c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=33, maxMergeAtOnceExplicit=36, maxMergedSegmentMB=0.69921875, floorSegmentMB=2.064453125, forceMergeDeletesPctAllowed=29.34568317668536, segmentsPerTier=24.0, maxCFSSegmentSizeMB=1.58984375, noCFSRatio=0.0
[junit4] 2> 702420 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:43242 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-041e987e-c3e7-4367-88c8-f8e35e37db4c:NORMAL:127.0.0.1:46619|RBW], ReplicaUC[[DISK]DS-8a3641f0-6e3c-4270-847c-c5375c7ad87c:NORMAL:127.0.0.1:43242|RBW]]} size 0
[junit4] 2> 702466 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:46619 is added to blk_1073741825_1001 size 71
[junit4] 2> 702780 WARN (coreLoadExecutor-922-thread-1-processing-n:127.0.0.1:34034_xt) [n:127.0.0.1:34034_xt c:control_collection x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 703137 INFO (coreLoadExecutor-922-thread-1-processing-n:127.0.0.1:34034_xt) [n:127.0.0.1:34034_xt c:control_collection x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 703137 INFO (coreLoadExecutor-922-thread-1-processing-n:127.0.0.1:34034_xt) [n:127.0.0.1:34034_xt c:control_collection x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 703137 INFO (coreLoadExecutor-922-thread-1-processing-n:127.0.0.1:34034_xt) [n:127.0.0.1:34034_xt c:control_collection x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 703427 INFO (coreLoadExecutor-922-thread-1-processing-n:127.0.0.1:34034_xt) [n:127.0.0.1:34034_xt c:control_collection x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 703427 INFO (coreLoadExecutor-922-thread-1-processing-n:127.0.0.1:34034_xt) [n:127.0.0.1:34034_xt c:control_collection x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 703463 INFO (coreLoadExecutor-922-thread-1-processing-n:127.0.0.1:34034_xt) [n:127.0.0.1:34034_xt c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=911256050, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 704158 INFO (coreLoadExecutor-922-thread-1-processing-n:127.0.0.1:34034_xt) [n:127.0.0.1:34034_xt c:control_collection x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@734ce921[collection1] main]
[junit4] 2> 704211 INFO (coreLoadExecutor-922-thread-1-processing-n:127.0.0.1:34034_xt) [n:127.0.0.1:34034_xt c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 704227 INFO (coreLoadExecutor-922-thread-1-processing-n:127.0.0.1:34034_xt) [n:127.0.0.1:34034_xt c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 704229 INFO (coreLoadExecutor-922-thread-1-processing-n:127.0.0.1:34034_xt) [n:127.0.0.1:34034_xt c:control_collection x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 704266 INFO (searcherExecutor-923-thread-1-processing-n:127.0.0.1:34034_xt x:collection1 c:control_collection) [n:127.0.0.1:34034_xt c:control_collection x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@734ce921[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 704340 INFO (coreLoadExecutor-922-thread-1-processing-n:127.0.0.1:34034_xt) [n:127.0.0.1:34034_xt c:control_collection x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1581426742297362432
[junit4] 2> 704431 INFO (coreZkRegister-915-thread-1-processing-n:127.0.0.1:34034_xt x:collection1 c:control_collection) [n:127.0.0.1:34034_xt c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 704431 INFO (coreZkRegister-915-thread-1-processing-n:127.0.0.1:34034_xt x:collection1 c:control_collection) [n:127.0.0.1:34034_xt c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 704431 INFO (coreZkRegister-915-thread-1-processing-n:127.0.0.1:34034_xt x:collection1 c:control_collection) [n:127.0.0.1:34034_xt c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:34034/xt/collection1/
[junit4] 2> 704431 INFO (coreZkRegister-915-thread-1-processing-n:127.0.0.1:34034_xt x:collection1 c:control_collection) [n:127.0.0.1:34034_xt c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 704431 INFO (coreZkRegister-915-thread-1-processing-n:127.0.0.1:34034_xt x:collection1 c:control_collection) [n:127.0.0.1:34034_xt c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:34034/xt/collection1/ has no replicas
[junit4] 2> 704431 INFO (coreZkRegister-915-thread-1-processing-n:127.0.0.1:34034_xt x:collection1 c:control_collection) [n:127.0.0.1:34034_xt c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 704433 INFO (coreZkRegister-915-thread-1-processing-n:127.0.0.1:34034_xt x:collection1 c:control_collection) [n:127.0.0.1:34034_xt c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:34034/xt/collection1/ shard1
[junit4] 2> 704653 INFO (coreZkRegister-915-thread-1-processing-n:127.0.0.1:34034_xt x:collection1 c:control_collection) [n:127.0.0.1:34034_xt c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 704984 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 704985 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37605/solr ready
[junit4] 2> 704985 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 705656 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_E1BDE91CA304AC18-001/shard-1-001/cores/collection1
[junit4] 2> 705657 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_E1BDE91CA304AC18-001/shard-1-001
[junit4] 2> 705674 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 705792 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@54b73292{/xt,null,AVAILABLE}
[junit4] 2> 705792 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@4a08147b{HTTP/1.1,[http/1.1]}{127.0.0.1:45253}
[junit4] 2> 705792 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.e.j.s.Server Started @719664ms
[junit4] 2> 705792 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://lucene2-us-west.apache.org:42643/hdfs__lucene2-us-west.apache.org_42643__home_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-6.6_checkout_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.StressHdfsTest_E1BDE91CA304AC18-001_tempDir-002_jetty1, solrconfig=solrconfig.xml, hostContext=/xt, hostPort=45253, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_E1BDE91CA304AC18-001/shard-1-001/cores}
[junit4] 2> 705792 ERROR (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 705808 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.6.2
[junit4] 2> 705808 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 705808 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 705808 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-10-16T15:01:08.875Z
[junit4] 2> 705918 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 705918 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_E1BDE91CA304AC18-001/shard-1-001/solr.xml
[junit4] 2> 705975 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 705991 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37605/solr
[junit4] 2> 706246 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [n:127.0.0.1:45253_xt ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 706281 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [n:127.0.0.1:45253_xt ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 706282 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [n:127.0.0.1:45253_xt ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45253_xt
[junit4] 2> 706338 INFO (zkCallback-271-thread-1-processing-n:127.0.0.1:45253_xt) [n:127.0.0.1:45253_xt ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 706393 INFO (zkCallback-262-thread-2-processing-n:127.0.0.1:34034_xt) [n:127.0.0.1:34034_xt ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 706394 INFO (zkCallback-266-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 707421 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [n:127.0.0.1:45253_xt ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_E1BDE91CA304AC18-001/shard-1-001/cores
[junit4] 2> 707421 INFO (TEST-StressHdfsTest.test-seed#[E1BDE91CA304AC18]) [n:127.0.0.1:45253_xt ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 707552 INFO (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:45253_xt) [n:127.0.0.1:45253_xt ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 707601 INFO (OverseerStateUpdate-98839170572484612-127.0.0.1:34034_xt-n_0000000000) [n:127.0.0.1:34034_xt ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 708881 INFO (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:45253_xt) [n:127.0.0.1:45253_xt c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.2
[junit4] 2> 709019 INFO (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:45253_xt) [n:127.0.0.1:45253_xt c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 710139 INFO (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:45253_xt) [n:127.0.0.1:45253_xt c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 710214 INFO (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:45253_xt) [n:127.0.0.1:45253_xt c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1, trusted=true
[junit4] 2> 710214 INFO (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:45253_xt) [n:127.0.0.1:45253_xt c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://lucene2-us-west.apache.org:42643/solr_hdfs_home
[junit4] 2> 710214 INFO (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:45253_xt) [n:127.0.0.1:45253_xt c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 710267 INFO (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:45253_xt) [n:127.0.0.1:45253_xt c:collection1 x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 710267 INFO (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:45253_xt) [n:127.0.0.1:45253_xt c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.StressHdfsTest_E1BDE91CA304AC18-001/shard-1-001/cores/collection1], dataDir=[hdfs://lucene2-us-west.apache.org:42643/solr_hdfs_home/collection1/core_node1/data/]
[junit4] 2> 710268 INFO (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:45253_xt) [n:127.0.0.1:45253_xt c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@616b5c77
[junit4] 2> 710268 INFO (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:45253_xt) [n:127.0.0.1:45253_xt c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:42643/solr_hdfs_home/collection1/core_node1/data/snapshot_metadata
[junit4] 2> 710307 INFO (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:45253_xt) [n:127.0.0.1:45253_xt c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 710307 INFO (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:45253_xt) [n:127.0.0.1:45253_xt c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 710307 INFO (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:45253_xt) [n:127.0.0.1:45253_xt c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 710523 INFO (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:45253_xt) [n:127.0.0.1:45253_xt c:collection1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 710539 INFO (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:45253_xt) [n:127.0.0.1:45253_xt c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:42643/solr_hdfs_home/collection1/core_node1/data
[junit4] 2> 710816 INFO (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:45253_xt) [n:127.0.0.1:45253_xt c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:42643/solr_hdfs_home/collection1/core_node1/data/index
[junit4] 2> 710856 INFO (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:45253_xt) [n:127.0.0.1:45253_xt c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 710856 INFO (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:45253_xt) [n:127.0.0.1:45253_xt c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 710856 INFO (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:45253_xt) [n:127.0.0.1:45253_xt c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 711111 INFO (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:45253_xt) [n:127.0.0.1:45253_xt c:collection1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 711111 INFO (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:45253_xt) [n:127.0.0.1:45253_xt c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=33, maxMergeAtOnceExplicit=36, maxMergedSegmentMB=0.69921875, floorSegmentMB=2.064453125, forceMergeDeletesPctAllowed=29.34568317668536, segmentsPerTier=24.0, maxCFSSegmentSizeMB=1.58984375, noCFSRatio=0.0
[junit4] 2> 712117 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:43242 is added to blk_1073741826_1002{UCState=COMMITTED, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-02980c51-d5ca-436f-8574-3f8813f8160a:NORMAL:127.0.0.1:46619|RBW], ReplicaUC[[DISK]DS-35f6830d-12df-49db-a539-fbf83a144630:NORMAL:127.0.0.1:43242|RBW]]} size 71
[junit4] 2> 712153 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:46619 is added to blk_1073741826_1002 size 71
[junit4] 2> 712683 WARN (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:45253_xt) [n:127.0.0.1:45253_xt c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 713182 INFO (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:45253_xt) [n:127.0.0.1:45253_xt c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 713182 INFO (coreLoadExecutor-933-thread-1-processing-n:127.0.0.1:45253_xt) [n:127.0.0.1:45253_xt c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 713182 INFO (coreL
[...truncated too long message...]
junit4] 2>
[junit4] 2> 12 /solr/collections
[junit4] 2> 7 /solr/live_nodes
[junit4] 2> 5 /solr/overseer/queue
[junit4] 2> 5 /solr/overseer/queue-work
[junit4] 2> 5 /solr/overseer/collection-queue-work
[junit4] 2>
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=SharedFSAutoReplicaFailoverTest -Dtests.method=test -Dtests.seed=E1BDE91CA304AC18 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/test-data/enwiki.random.lines.txt -Dtests.locale=es-NI -Dtests.timezone=Africa/Mogadishu -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] FAILURE 287s J0 | SharedFSAutoReplicaFailoverTest.test <<<
[junit4] > Throwable #1: java.lang.AssertionError: expected:<2> but was:<4>
[junit4] > at __randomizedtesting.SeedInfo.seed([E1BDE91CA304AC18:69E9D6C60DF8C1E0]:0)
[junit4] > at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.assertSliceAndReplicaCount(SharedFSAutoReplicaFailoverTest.java:419)
[junit4] > at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.testBasics(SharedFSAutoReplicaFailoverTest.java:264)
[junit4] > at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test(SharedFSAutoReplicaFailoverTest.java:136)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 9413099 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[E1BDE91CA304AC18]-worker) [ ] o.a.h.h.s.d.DirectoryScanner DirectoryScanner: shutdown has been called
[junit4] 2> 9413189 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[E1BDE91CA304AC18]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
[junit4] 2> 9413296 WARN (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_E1BDE91CA304AC18-001/tempDir-001/hdfsBaseDir/data/data3/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_E1BDE91CA304AC18-001/tempDir-001/hdfsBaseDir/data/data4/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:35596) [ ] o.a.h.h.s.d.IncrementalBlockReportManager IncrementalBlockReportManager interrupted
[junit4] 2> 9413297 WARN (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_E1BDE91CA304AC18-001/tempDir-001/hdfsBaseDir/data/data3/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_E1BDE91CA304AC18-001/tempDir-001/hdfsBaseDir/data/data4/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:35596) [ ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-398306373-127.0.0.1-1508174479274 (Datanode Uuid 03bd5b2d-8c5e-4209-9223-a451b18458eb) service to lucene2-us-west.apache.org/127.0.0.1:35596
[junit4] 2> 9413298 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[E1BDE91CA304AC18]-worker) [ ] o.a.h.h.s.d.DirectoryScanner DirectoryScanner: shutdown has been called
[junit4] 2> 9413509 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[E1BDE91CA304AC18]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
[junit4] 2> 9413618 WARN (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_E1BDE91CA304AC18-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_E1BDE91CA304AC18-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:35596) [ ] o.a.h.h.s.d.IncrementalBlockReportManager IncrementalBlockReportManager interrupted
[junit4] 2> 9413618 WARN (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_E1BDE91CA304AC18-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_E1BDE91CA304AC18-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:35596) [ ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-398306373-127.0.0.1-1508174479274 (Datanode Uuid 1cd82927-4376-4d01-9ccf-59238d3efd87) service to lucene2-us-west.apache.org/127.0.0.1:35596
[junit4] 2> 9413919 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[E1BDE91CA304AC18]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@lucene2-us-west.apache.org:0
[junit4] 2> 9414021 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[E1BDE91CA304AC18]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_E1BDE91CA304AC18-001
[junit4] 2> Oct 16, 2017 5:26:17 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 34 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): {id=PostingsFormat(name=MockRandom), text=PostingsFormat(name=LuceneFixedGap)}, docValues:{_version_=DocValuesFormat(name=Direct)}, maxPointsInLeafNode=25, maxMBSortInHeap=5.120041633880155, sim=RandomSimilarity(queryNorm=true,coord=crazy): {}, locale=es-NI, timezone=Africa/Mogadishu
[junit4] 2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=2,free=234854624,total=533725184
[junit4] 2> NOTE: All tests run in this JVM: [TestDelegationWithHadoopAuth, FileBasedSpellCheckerTest, QueryElevationComponentTest, TestPointFields, TestCloudSchemaless, TestReplicationHandler, BasicDistributedZkTest, TestHighlightDedupGrouping, TestRangeQuery, TestApiFramework, InfoHandlerTest, CustomCollectionTest, ReplicationFactorTest, TestPerFieldSimilarity, CoreAdminCreateDiscoverTest, TestMinMaxOnMultiValuedField, TestSolrCLIRunExample, TestAtomicUpdateErrorCases, HdfsChaosMonkeySafeLeaderTest, StandardRequestHandlerTest, TestNestedDocsSort, TestPKIAuthenticationPlugin, CacheHeaderTest, TestComplexPhraseQParserPlugin, BigEndianAscendingWordDeserializerTest, TestClassNameShortening, SolrCLIZkUtilsTest, ConfigureRecoveryStrategyTest, TestCoreAdminApis, DisMaxRequestHandlerTest, TestStressLiveNodes, LeaderInitiatedRecoveryOnCommitTest, TestRealTimeGet, TestCloudJSONFacetJoinDomain, SecurityConfHandlerTest, TestBackupRepositoryFactory, TestPHPSerializedResponseWriter, TestPayloadCheckQParserPlugin, XsltUpdateRequestHandlerTest, TestSolrCloudSnapshots, AnalysisAfterCoreReloadTest, TestUseDocValuesAsStored, TestLazyCores, DistributedSpellCheckComponentTest, TestOmitPositions, BufferStoreTest, BasicAuthStandaloneTest, TestCryptoKeys, TestFieldCacheSanityChecker, TestDistributedSearch, TestSolrDeletionPolicy2, TestReplicaProperties, TestCloudRecovery, TestLeaderElectionWithEmptyReplica, BaseCdcrDistributedZkTest, ManagedSchemaRoundRobinCloudTest, ConjunctionSolrSpellCheckerTest, TestPathTrie, VersionInfoTest, DateFieldTest, TestAuthenticationFramework, TestAddFieldRealTimeGet, ExitableDirectoryReaderTest, TestReloadAndDeleteDocs, TestDefaultStatsCache, TestSimpleQParserPlugin, MissingSegmentRecoveryTest, TestBinaryResponseWriter, NumericFieldsTest, TestBadConfig, BlobRepositoryCloudTest, DocumentAnalysisRequestHandlerTest, OverseerCollectionConfigSetProcessorTest, TestExactStatsCache, HdfsSyncSliceTest, TestSolr4Spatial, PathHierarchyTokenizerFactoryTest, TestRecoveryHdfs, TestValueSourceCache, TestHighFrequencyDictionaryFactory, ShufflingReplicaListTransformerTest, TestJettySolrRunner, JvmMetricsTest, JsonValidatorTest, TestBlobHandler, TestClassicSimilarityFactory, AtomicUpdatesTest, SolrCoreTest, SolrGraphiteReporterTest, SSLMigrationTest, TestMultiWordSynonyms, ResourceLoaderTest, SynonymTokenizerTest, SyncSliceTest, BasicZkTest, TestQuerySenderNoQuery, FacetPivotSmallTest, HdfsUnloadDistributedZkTest, DateRangeFieldTest, TestPerFieldSimilarityClassic, TestCSVLoader, TestInPlaceUpdatesDistrib, TestCrossCoreJoin, TestSolrCloudWithSecureImpersonation, DistributedFacetExistsSmallTest, TestDownShardTolerantSearch, TestFunctionQuery, TestDynamicFieldResource, UniqFieldsUpdateProcessorFactoryTest, TestDocTermOrds, SolrSlf4jReporterTest, TestDocBasedVersionConstraints, TestManagedSchema, TestPseudoReturnFields, PolyFieldTest, CdcrVersionReplicationTest, CollectionReloadTest, CollectionsAPIAsyncDistributedZkTest, ConfigSetsAPITest, ConnectionManagerTest, CreateCollectionCleanupTest, DeleteNodeTest, DeleteStatusTest, DistribJoinFromCollectionTest, HttpPartitionTest, NodeMutatorTest, OverseerRolesTest, OverseerStatusTest, ReplaceNodeTest, SaslZkACLProviderTest, SharedFSAutoReplicaFailoverTest]
[junit4] Completed [538/713 (5!)] on J0 in 307.83s, 1 test, 1 failure <<< FAILURES!
[...truncated 44075 lines...]
[JENKINS] Lucene-Solr-NightlyTests-6.6 - Build # 30 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.6/30/
2 tests failed.
FAILED: org.apache.solr.cloud.CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap
Error Message:
Document mismatch on target after sync expected:<10000> but was:<0>
Stack Trace:
java.lang.AssertionError: Document mismatch on target after sync expected:<10000> but was:<0>
at __randomizedtesting.SeedInfo.seed([8F84846AADE38943:5853AB1D19BC1104]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.apache.solr.cloud.CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap(CdcrBootstrapTest.java:134)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test
Error Message:
expected:<2> but was:<3>
Stack Trace:
java.lang.AssertionError: expected:<2> but was:<3>
at __randomizedtesting.SeedInfo.seed([8F84846AADE38943:7D0BBB0031FE4BB]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.assertSliceAndReplicaCount(SharedFSAutoReplicaFailoverTest.java:419)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.testBasics(SharedFSAutoReplicaFailoverTest.java:264)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test(SharedFSAutoReplicaFailoverTest.java:136)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 11836 lines...]
[junit4] Suite: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_8F84846AADE38943-001/init-core-data-001
[junit4] 2> 203743 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[8F84846AADE38943]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=4 numCloses=4
[junit4] 2> 203743 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[8F84846AADE38943]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields
[junit4] 2> 203744 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[8F84846AADE38943]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=None)
[junit4] 2> 203744 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[8F84846AADE38943]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /v/e
[junit4] 2> 205957 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[8F84846AADE38943]-worker) [ ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 209544 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[8F84846AADE38943]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 210372 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[8F84846AADE38943]-worker) [ ] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
[junit4] 2> 210441 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[8F84846AADE38943]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 212442 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[8F84846AADE38943]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 212754 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[8F84846AADE38943]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_47452_hdfs____6nhw60/webapp
[junit4] 2> 214467 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[8F84846AADE38943]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:47452
[junit4] 2> 220090 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[8F84846AADE38943]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 220097 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[8F84846AADE38943]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 220265 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[8F84846AADE38943]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_53521_datanode____.ss8ncz/webapp
[junit4] 2> 221091 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[8F84846AADE38943]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:53521
[junit4] 2> 223712 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[8F84846AADE38943]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 223724 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[8F84846AADE38943]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 223777 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[8F84846AADE38943]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_43792_datanode____6cln2w/webapp
[junit4] 2> 226783 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[8F84846AADE38943]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43792
[junit4] 2> 227985 ERROR (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_8F84846AADE38943-001/tempDir-001/hdfsBaseDir/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_8F84846AADE38943-001/tempDir-001/hdfsBaseDir/data/data4/]] heartbeating to localhost/127.0.0.1:57357) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 228012 ERROR (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_8F84846AADE38943-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_8F84846AADE38943-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to localhost/127.0.0.1:57357) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 228456 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x1125e5856866ca: from storage DS-771019a8-18cc-4759-8154-7923750f1dfc node DatanodeRegistration(127.0.0.1:56865, datanodeUuid=59a0de29-1473-4c2a-aaf8-15aab33d829f, infoPort=58171, infoSecurePort=0, ipcPort=40590, storageInfo=lv=-56;cid=testClusterID;nsid=1684133339;c=0), blocks: 0, hasStaleStorage: true, processing time: 2 msecs
[junit4] 2> 228457 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x1125e5856866ca: from storage DS-81f269a9-ef1f-4e7f-bbae-824789291437 node DatanodeRegistration(127.0.0.1:56865, datanodeUuid=59a0de29-1473-4c2a-aaf8-15aab33d829f, infoPort=58171, infoSecurePort=0, ipcPort=40590, storageInfo=lv=-56;cid=testClusterID;nsid=1684133339;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 228467 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x1125e584f654c1: from storage DS-6f1867bb-3c3b-49a1-8466-76f7bd52f547 node DatanodeRegistration(127.0.0.1:43363, datanodeUuid=b3b17c42-29b9-4800-afdc-ac499ac342de, infoPort=33861, infoSecurePort=0, ipcPort=40847, storageInfo=lv=-56;cid=testClusterID;nsid=1684133339;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 228467 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x1125e584f654c1: from storage DS-151a7f34-1006-4de7-b6ea-e4935f23f6e5 node DatanodeRegistration(127.0.0.1:43363, datanodeUuid=b3b17c42-29b9-4800-afdc-ac499ac342de, infoPort=33861, infoSecurePort=0, ipcPort=40847, storageInfo=lv=-56;cid=testClusterID;nsid=1684133339;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 231545 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 231562 INFO (Thread-260) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 231562 INFO (Thread-260) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 231602 ERROR (Thread-260) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 231730 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.c.ZkTestServer start zk server on port:59534
[junit4] 2> 231844 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 231846 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 231847 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 231848 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 231849 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 231850 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 231851 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 231852 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 231853 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 231854 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 231855 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 232328 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_8F84846AADE38943-001/control-001/cores/collection1
[junit4] 2> 232358 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 232462 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@e65b7c7{/v/e,null,AVAILABLE}
[junit4] 2> 232462 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@65618004{HTTP/1.1,[http/1.1]}{127.0.0.1:58634}
[junit4] 2> 232463 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.e.j.s.Server Started @240616ms
[junit4] 2> 232463 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/v/e, hostPort=58634, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_8F84846AADE38943-001/control-001/cores}
[junit4] 2> 232463 ERROR (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 232463 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.6.1
[junit4] 2> 232463 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 232463 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 232463 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-09-05T01:26:58.576Z
[junit4] 2> 232506 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 232507 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_8F84846AADE38943-001/control-001/solr.xml
[junit4] 2> 232542 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 232554 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:59534/solr
[junit4] 2> 232626 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [n:127.0.0.1:58634_v%2Fe ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 232627 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [n:127.0.0.1:58634_v%2Fe ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:58634_v%2Fe
[junit4] 2> 232628 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [n:127.0.0.1:58634_v%2Fe ] o.a.s.c.Overseer Overseer (id=98603815251673092-127.0.0.1:58634_v%2Fe-n_0000000000) starting
[junit4] 2> 232769 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [n:127.0.0.1:58634_v%2Fe ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58634_v%2Fe
[junit4] 2> 232774 INFO (OverseerStateUpdate-98603815251673092-127.0.0.1:58634_v%2Fe-n_0000000000) [n:127.0.0.1:58634_v%2Fe ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 233095 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [n:127.0.0.1:58634_v%2Fe ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_8F84846AADE38943-001/control-001/cores
[junit4] 2> 233096 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [n:127.0.0.1:58634_v%2Fe ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 233126 INFO (coreLoadExecutor-320-thread-1-processing-n:127.0.0.1:58634_v%2Fe) [n:127.0.0.1:58634_v%2Fe c:control_collection x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 233135 INFO (OverseerStateUpdate-98603815251673092-127.0.0.1:58634_v%2Fe-n_0000000000) [n:127.0.0.1:58634_v%2Fe ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 233197 WARN (coreLoadExecutor-320-thread-1-processing-n:127.0.0.1:58634_v%2Fe) [n:127.0.0.1:58634_v%2Fe c:control_collection x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 233198 INFO (coreLoadExecutor-320-thread-1-processing-n:127.0.0.1:58634_v%2Fe) [n:127.0.0.1:58634_v%2Fe c:control_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.1
[junit4] 2> 233298 INFO (coreLoadExecutor-320-thread-1-processing-n:127.0.0.1:58634_v%2Fe) [n:127.0.0.1:58634_v%2Fe c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 233679 INFO (coreLoadExecutor-320-thread-1-processing-n:127.0.0.1:58634_v%2Fe) [n:127.0.0.1:58634_v%2Fe c:control_collection x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 233723 INFO (coreLoadExecutor-320-thread-1-processing-n:127.0.0.1:58634_v%2Fe) [n:127.0.0.1:58634_v%2Fe c:control_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection, trusted=true
[junit4] 2> 233830 INFO (coreLoadExecutor-320-thread-1-processing-n:127.0.0.1:58634_v%2Fe) [n:127.0.0.1:58634_v%2Fe c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:57357/solr_hdfs_home
[junit4] 2> 233830 INFO (coreLoadExecutor-320-thread-1-processing-n:127.0.0.1:58634_v%2Fe) [n:127.0.0.1:58634_v%2Fe c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 233830 INFO (coreLoadExecutor-320-thread-1-processing-n:127.0.0.1:58634_v%2Fe) [n:127.0.0.1:58634_v%2Fe c:control_collection x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 233830 INFO (coreLoadExecutor-320-thread-1-processing-n:127.0.0.1:58634_v%2Fe) [n:127.0.0.1:58634_v%2Fe c:control_collection x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_8F84846AADE38943-001/control-001/cores/collection1], dataDir=[hdfs://localhost:57357/solr_hdfs_home/control_collection/core_node1/data/]
[junit4] 2> 233831 INFO (coreLoadExecutor-320-thread-1-processing-n:127.0.0.1:58634_v%2Fe) [n:127.0.0.1:58634_v%2Fe c:control_collection x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@41c15ff1
[junit4] 2> 233831 INFO (coreLoadExecutor-320-thread-1-processing-n:127.0.0.1:58634_v%2Fe) [n:127.0.0.1:58634_v%2Fe c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:57357/solr_hdfs_home/control_collection/core_node1/data/snapshot_metadata
[junit4] 2> 233890 INFO (coreLoadExecutor-320-thread-1-processing-n:127.0.0.1:58634_v%2Fe) [n:127.0.0.1:58634_v%2Fe c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 233891 INFO (coreLoadExecutor-320-thread-1-processing-n:127.0.0.1:58634_v%2Fe) [n:127.0.0.1:58634_v%2Fe c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
[junit4] 2> 233891 INFO (coreLoadExecutor-320-thread-1-processing-n:127.0.0.1:58634_v%2Fe) [n:127.0.0.1:58634_v%2Fe c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new global HDFS BlockCache
[junit4] 2> 235825 INFO (coreLoadExecutor-320-thread-1-processing-n:127.0.0.1:58634_v%2Fe) [n:127.0.0.1:58634_v%2Fe c:control_collection x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 235831 INFO (coreLoadExecutor-320-thread-1-processing-n:127.0.0.1:58634_v%2Fe) [n:127.0.0.1:58634_v%2Fe c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:57357/solr_hdfs_home/control_collection/core_node1/data
[junit4] 2> 235982 INFO (coreLoadExecutor-320-thread-1-processing-n:127.0.0.1:58634_v%2Fe) [n:127.0.0.1:58634_v%2Fe c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:57357/solr_hdfs_home/control_collection/core_node1/data/index
[junit4] 2> 236005 INFO (coreLoadExecutor-320-thread-1-processing-n:127.0.0.1:58634_v%2Fe) [n:127.0.0.1:58634_v%2Fe c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 236005 INFO (coreLoadExecutor-320-thread-1-processing-n:127.0.0.1:58634_v%2Fe) [n:127.0.0.1:58634_v%2Fe c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
[junit4] 2> 236021 INFO (coreLoadExecutor-320-thread-1-processing-n:127.0.0.1:58634_v%2Fe) [n:127.0.0.1:58634_v%2Fe c:control_collection x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 236021 INFO (coreLoadExecutor-320-thread-1-processing-n:127.0.0.1:58634_v%2Fe) [n:127.0.0.1:58634_v%2Fe c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=2065041584, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 237121 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:56865 is added to blk_1073741825_1001{UCState=COMMITTED, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-81f269a9-ef1f-4e7f-bbae-824789291437:NORMAL:127.0.0.1:56865|RBW], ReplicaUC[[DISK]DS-6f1867bb-3c3b-49a1-8466-76f7bd52f547:NORMAL:127.0.0.1:43363|RBW]]} size 71
[junit4] 2> 237124 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:43363 is added to blk_1073741825_1001 size 71
[junit4] 2> 237595 WARN (coreLoadExecutor-320-thread-1-processing-n:127.0.0.1:58634_v%2Fe) [n:127.0.0.1:58634_v%2Fe c:control_collection x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 238034 INFO (coreLoadExecutor-320-thread-1-processing-n:127.0.0.1:58634_v%2Fe) [n:127.0.0.1:58634_v%2Fe c:control_collection x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 238034 INFO (coreLoadExecutor-320-thread-1-processing-n:127.0.0.1:58634_v%2Fe) [n:127.0.0.1:58634_v%2Fe c:control_collection x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 238034 INFO (coreLoadExecutor-320-thread-1-processing-n:127.0.0.1:58634_v%2Fe) [n:127.0.0.1:58634_v%2Fe c:control_collection x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 238067 INFO (coreLoadExecutor-320-thread-1-processing-n:127.0.0.1:58634_v%2Fe) [n:127.0.0.1:58634_v%2Fe c:control_collection x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 238067 INFO (coreLoadExecutor-320-thread-1-processing-n:127.0.0.1:58634_v%2Fe) [n:127.0.0.1:58634_v%2Fe c:control_collection x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 238075 INFO (coreLoadExecutor-320-thread-1-processing-n:127.0.0.1:58634_v%2Fe) [n:127.0.0.1:58634_v%2Fe c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=33, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4500553319634587]
[junit4] 2> 238555 INFO (coreLoadExecutor-320-thread-1-processing-n:127.0.0.1:58634_v%2Fe) [n:127.0.0.1:58634_v%2Fe c:control_collection x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2e211510[collection1] main]
[junit4] 2> 238556 INFO (coreLoadExecutor-320-thread-1-processing-n:127.0.0.1:58634_v%2Fe) [n:127.0.0.1:58634_v%2Fe c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 238557 INFO (coreLoadExecutor-320-thread-1-processing-n:127.0.0.1:58634_v%2Fe) [n:127.0.0.1:58634_v%2Fe c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 238559 INFO (coreLoadExecutor-320-thread-1-processing-n:127.0.0.1:58634_v%2Fe) [n:127.0.0.1:58634_v%2Fe c:control_collection x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 238560 INFO (searcherExecutor-321-thread-1-processing-n:127.0.0.1:58634_v%2Fe x:collection1 c:control_collection) [n:127.0.0.1:58634_v%2Fe c:control_collection x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2e211510[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 238572 INFO (coreLoadExecutor-320-thread-1-processing-n:127.0.0.1:58634_v%2Fe) [n:127.0.0.1:58634_v%2Fe c:control_collection x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577661051368898560
[junit4] 2> 238603 INFO (coreZkRegister-313-thread-1-processing-n:127.0.0.1:58634_v%2Fe x:collection1 c:control_collection) [n:127.0.0.1:58634_v%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 238603 INFO (coreZkRegister-313-thread-1-processing-n:127.0.0.1:58634_v%2Fe x:collection1 c:control_collection) [n:127.0.0.1:58634_v%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 238603 INFO (coreZkRegister-313-thread-1-processing-n:127.0.0.1:58634_v%2Fe x:collection1 c:control_collection) [n:127.0.0.1:58634_v%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:58634/v/e/collection1/
[junit4] 2> 238603 INFO (coreZkRegister-313-thread-1-processing-n:127.0.0.1:58634_v%2Fe x:collection1 c:control_collection) [n:127.0.0.1:58634_v%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 238603 INFO (coreZkRegister-313-thread-1-processing-n:127.0.0.1:58634_v%2Fe x:collection1 c:control_collection) [n:127.0.0.1:58634_v%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:58634/v/e/collection1/ has no replicas
[junit4] 2> 238603 INFO (coreZkRegister-313-thread-1-processing-n:127.0.0.1:58634_v%2Fe x:collection1 c:control_collection) [n:127.0.0.1:58634_v%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 238607 INFO (coreZkRegister-313-thread-1-processing-n:127.0.0.1:58634_v%2Fe x:collection1 c:control_collection) [n:127.0.0.1:58634_v%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:58634/v/e/collection1/ shard1
[junit4] 2> 238850 INFO (coreZkRegister-313-thread-1-processing-n:127.0.0.1:58634_v%2Fe x:collection1 c:control_collection) [n:127.0.0.1:58634_v%2Fe c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 238853 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 238854 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:59534/solr ready
[junit4] 2> 238856 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 238856 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
[junit4] 2> 239113 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_8F84846AADE38943-001/shard-1-001/cores/collection1
[junit4] 2> 239113 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_8F84846AADE38943-001/shard-1-001
[junit4] 2> 239115 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 239266 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@398b1187{/v/e,null,AVAILABLE}
[junit4] 2> 239267 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@28256ce{HTTP/1.1,[http/1.1]}{127.0.0.1:60477}
[junit4] 2> 239267 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.e.j.s.Server Started @247420ms
[junit4] 2> 239267 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml, hostContext=/v/e, hostPort=60477, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_8F84846AADE38943-001/shard-1-001/cores}
[junit4] 2> 239267 ERROR (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 239279 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.6.1
[junit4] 2> 239280 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 239280 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 239280 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-09-05T01:27:05.393Z
[junit4] 2> 239348 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 239348 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_8F84846AADE38943-001/shard-1-001/solr.xml
[junit4] 2> 239361 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 239361 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:59534/solr
[junit4] 2> 239464 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [n:127.0.0.1:60477_v%2Fe ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 239467 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [n:127.0.0.1:60477_v%2Fe ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 239470 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [n:127.0.0.1:60477_v%2Fe ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60477_v%2Fe
[junit4] 2> 239471 INFO (zkCallback-95-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 239471 INFO (zkCallback-91-thread-2-processing-n:127.0.0.1:58634_v%2Fe) [n:127.0.0.1:58634_v%2Fe ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 239484 INFO (zkCallback-101-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 239974 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [n:127.0.0.1:60477_v%2Fe ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_8F84846AADE38943-001/shard-1-001/cores
[junit4] 2> 239974 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [n:127.0.0.1:60477_v%2Fe ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 240028 INFO (coreLoadExecutor-331-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe c:collection1 x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 240032 INFO (OverseerStateUpdate-98603815251673092-127.0.0.1:58634_v%2Fe-n_0000000000) [n:127.0.0.1:58634_v%2Fe ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 240134 INFO (zkCallback-101-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 241073 WARN (coreLoadExecutor-331-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe c:collection1 x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 241091 INFO (coreLoadExecutor-331-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.1
[junit4] 2> 241196 INFO (coreLoadExecutor-331-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 241738 INFO (coreLoadExecutor-331-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 242809 INFO (coreLoadExecutor-331-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1, trusted=true
[junit4] 2> 242810 INFO (coreLoadExecutor-331-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:57357/solr_hdfs_home
[junit4] 2> 242810 INFO (coreLoadExecutor-331-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 242810 INFO (coreLoadExecutor-331-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe c:collection1 x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 242810 INFO (coreLoadExecutor-331-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_8F84846AADE38943-001/shard-1-001/cores/collection1], dataDir=[hdfs://localhost:57357/solr_hdfs_home/collection1/core_node1/data/]
[junit4] 2> 242810 INFO (coreLoadExecutor-331-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@41c15ff1
[junit4] 2> 242810 INFO (coreLoadExecutor-331-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:57357/solr_hdfs_home/collection1/core_node1/data/snapshot_metadata
[junit4] 2> 242841 INFO (coreLoadExecutor-331-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 242841 INFO (coreLoadExecutor-331-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
[junit4] 2> 242873 INFO (coreLoadExecutor-331-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe c:collection1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 242874 INFO (coreLoadExecutor-331-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:57357/solr_hdfs_home/collection1/core_node1/data
[junit4] 2> 242940 INFO (coreLoadExecutor-331-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:57357/solr_hdfs_home/collection1/core_node1/data/index
[junit4] 2> 242950 INFO (coreLoadExecutor-331-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 242950 INFO (coreLoadExecutor-331-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
[junit4] 2> 242970 INFO (coreLoadExecutor-331-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe c:collection1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 242971 INFO (coreLoadExecutor-331-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=2065041584, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 243127 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:56865 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-151a7f34-1006-4de7-b6ea-e4935f23f6e5:NORMAL:127.0.0.1:43363|RBW], ReplicaUC[[DISK]DS-81f269a9-ef1f-4e7f-bbae-824789291437:NORMAL:127.0.0.1:56865|FINALIZED]]} size 0
[junit4] 2> 243157 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:43363 is added to blk_1073741826_1002 size 71
[junit4] 2> 243169 WARN (coreLoadExecutor-331-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 243451 INFO (coreLoadExecutor-331-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 243451 INFO (coreLoadExecutor-331-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 243451 INFO (coreLoadExecutor-331-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe c:collection1 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 243542 INFO (coreLoadExecutor-331-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 243542 INFO (coreLoadExecutor-331-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 243545 INFO (coreLoadExecutor-331-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=33, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4500553319634587]
[junit4] 2> 243639 INFO (coreLoadExecutor-331-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@7d17bcc5[collection1] main]
[junit4] 2> 243639 INFO (coreLoadExecutor-331-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 243640 INFO (coreLoadExecutor-331-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 243640 INFO (coreLoadExecutor-331-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 243642 INFO (searcherExecutor-332-thread-1-processing-n:127.0.0.1:60477_v%2Fe x:collection1 c:collection1) [n:127.0.0.1:60477_v%2Fe c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@7d17bcc5[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 243647 INFO (coreLoadExecutor-331-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577661056690421760
[junit4] 2> 243671 INFO (coreZkRegister-326-thread-1-processing-n:127.0.0.1:60477_v%2Fe x:collection1 c:collection1) [n:127.0.0.1:60477_v%2Fe c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 243671 INFO (coreZkRegister-326-thread-1-processing-n:127.0.0.1:60477_v%2Fe x:collection1 c:collection1) [n:127.0.0.1:60477_v%2Fe c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 243671 INFO (coreZkRegister-326-thread-1-processing-n:127.0.0.1:60477_v%2Fe x:collection1 c:collection1) [n:127.0.0.1:60477_v%2Fe c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:60477/v/e/collection1/
[junit4] 2> 243671 INFO (coreZkRegister-326-thread-1-processing-n:127.0.0.1:60477_v%2Fe x:collection1 c:collection1) [n:127.0.0.1:60477_v%2Fe c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 243671 INFO (coreZkRegister-326-thread-1-processing-n:127.0.0.1:60477_v%2Fe x:collection1 c:collection1) [n:127.0.0.1:60477_v%2Fe c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:60477/v/e/collection1/ has no replicas
[junit4] 2> 243671 INFO (coreZkRegister-326-thread-1-processing-n:127.0.0.1:60477_v%2Fe x:collection1 c:collection1) [n:127.0.0.1:60477_v%2Fe c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 243672 INFO (zkCallback-101-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 243675 INFO (coreZkRegister-326-thread-1-processing-n:127.0.0.1:60477_v%2Fe x:collection1 c:collection1) [n:127.0.0.1:60477_v%2Fe c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:60477/v/e/collection1/ shard2
[junit4] 2> 243815 INFO (zkCallback-101-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 243865 INFO (coreZkRegister-326-thread-1-processing-n:127.0.0.1:60477_v%2Fe x:collection1 c:collection1) [n:127.0.0.1:60477_v%2Fe c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 243970 INFO (zkCallback-101-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 244353 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_8F84846AADE38943-001/shard-2-001/cores/collection1
[junit4] 2> 244358 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_8F84846AADE38943-001/shard-2-001
[junit4] 2> 244359 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 244442 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4621cfe3{/v/e,null,AVAILABLE}
[junit4] 2> 244442 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3d5d79a8{HTTP/1.1,[http/1.1]}{127.0.0.1:57301}
[junit4] 2> 244442 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.e.j.s.Server Started @252596ms
[junit4] 2> 244443 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml, hostContext=/v/e, hostPort=57301, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_8F84846AADE38943-001/shard-2-001/cores}
[junit4] 2> 244443 ERROR (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 244451 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.6.1
[junit4] 2> 244451 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 244451 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 244451 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-09-05T01:27:10.564Z
[junit4] 2> 244526 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 244526 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_8F84846AADE38943-001/shard-2-001/solr.xml
[junit4] 2> 244547 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 244550 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:59534/solr
[junit4] 2> 244762 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [n:127.0.0.1:57301_v%2Fe ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 244765 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [n:127.0.0.1:57301_v%2Fe ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 244767 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [n:127.0.0.1:57301_v%2Fe ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57301_v%2Fe
[junit4] 2> 244774 INFO (zkCallback-101-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 244776 INFO (zkCallback-95-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 244774 INFO (zkCallback-91-thread-2-processing-n:127.0.0.1:58634_v%2Fe) [n:127.0.0.1:58634_v%2Fe ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 244778 INFO (zkCallback-107-thread-1-processing-n:127.0.0.1:57301_v%2Fe) [n:127.0.0.1:57301_v%2Fe ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 244953 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [n:127.0.0.1:57301_v%2Fe ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_8F84846AADE38943-001/shard-2-001/cores
[junit4] 2> 244953 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [n:127.0.0.1:57301_v%2Fe ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 245022 INFO (coreLoadExecutor-342-thread-1-processing-n:127.0.0.1:57301_v%2Fe) [n:127.0.0.1:57301_v%2Fe c:collection1 x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 245027 INFO (OverseerStateUpdate-98603815251673092-127.0.0.1:58634_v%2Fe-n_0000000000) [n:127.0.0.1:58634_v%2Fe ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 245144 INFO (zkCallback-101-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 245144 INFO (zkCallback-107-thread-1-processing-n:127.0.0.1:57301_v%2Fe) [n:127.0.0.1:57301_v%2Fe ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 246059 WARN (coreLoadExecutor-342-thread-1-processing-n:127.0.0.1:57301_v%2Fe) [n:127.0.0.1:57301_v%2Fe c:collection1 x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 246059 INFO (coreLoadExecutor-342-thread-1-processing-n:127.0.0.1:57301_v%2Fe) [n:127.0.0.1:57301_v%2Fe c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.1
[junit4] 2> 246211 INFO (coreLoadExecutor-342-thread-1-processing-n:127.0.0.1:57301_v%2Fe) [n:127.0.0.1:57301_v%2Fe c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 247152 INFO (coreLoadExecutor-342-thread-1-processing-n:127.0.0.1:57301_v%2Fe) [n:127.0.0.1:57301_v%2Fe c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 247201 INFO (coreLoadExecutor-342-thread-1-processing-n:127.0.0.1:57301_v%2Fe) [n:127.0.0.1:57301_v%2Fe c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1, trusted=true
[junit4] 2> 247220 INFO (coreLoadExecutor-342-thread-1-processing-n:127.0.0.1:57301_v%2Fe) [n:127.0.0.1:57301_v%2Fe c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:57357/solr_hdfs_home
[junit4] 2> 247220 INFO (coreLoadExecutor-342-thread-1-processing-n:127.0.0.1:57301_v%2Fe) [n:127.0.0.1:57301_v%2Fe c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 247220 INFO (coreLoadExecutor-342-thread-1-processing-n:127.0.0.1:57301_v%2Fe) [n:127.0.0.1:57301_v%2Fe c:collection1 x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 247220 INFO (coreLoadExecutor-342-thread-1-processing-n:127.0.0.1:57301_v%2Fe) [n:127.0.0.1:57301_v%2Fe c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_8F84846AADE38943-001/shard-2-001/cores/collection1], dataDir=[hdfs://localhost:57357/solr_hdfs_home/collection1/core_node2/data/]
[junit4] 2> 247220 INFO (coreLoadExecutor-342-thread-1-processing-n:127.0.0.1:57301_v%2Fe) [n:127.0.0.1:57301_v%2Fe c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@41c15ff1
[junit4] 2> 247221 INFO (coreLoadExecutor-342-thread-1-processing-n:127.0.0.1:57301_v%2Fe) [n:127.0.0.1:57301_v%2Fe c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:57357/solr_hdfs_home/collection1/core_node2/data/snapshot_metadata
[junit4] 2> 247234 INFO (coreLoadExecutor-342-thread-1-processing-n:127.0.0.1:57301_v%2Fe) [n:127.0.0.1:57301_v%2Fe c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 247235 INFO (coreLoadExecutor-342-thread-1-processing-n:127.0.0.1:57301_v%2Fe) [n:127.0.0.1:57301_v%2Fe c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
[junit4] 2> 247258 INFO (coreLoadExecutor-342-thread-1-processing-n:127.0.0.1:57301_v%2Fe) [n:127.0.0.1:57301_v%2Fe c:collection1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 247260 INFO (coreLoadExecutor-342-thread-1-processing-n:127.0.0.1:57301_v%2Fe) [n:127.0.0.1:57301_v%2Fe c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:57357/solr_hdfs_home/collection1/core_node2/data
[junit4] 2> 247368 INFO (coreLoadExecutor-342-thread-1-processing-n:127.0.0.1:57301_v%2Fe) [n:127.0.0.1:57301_v%2Fe c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:57357/solr_hdfs_home/collection1/core_node2/data/index
[junit4] 2> 247390 INFO (coreLoadExecutor-342-thread-1-processing-n:127.0.0.1:57301_v%2Fe) [n:127.0.0.1:57301_v%2Fe c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 247390 INFO (coreLoadExecutor-342-thread-1-processing-n:127.0.0.1:57301_v%2Fe) [n:127.0.0.1:57301_v%2Fe c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
[junit4] 2> 247405 INFO (coreLoadExecutor-342-thread-1-processing-n:127.0.0.1:57301_v%2Fe) [n:127.0.0.1:57301_v%2Fe c:collection1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 247406 INFO (coreLoadExecutor-342-thread-1-processing-n:127.0.0.1:57301_v%2Fe) [n:127.0.0.1:57301_v%2Fe c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=2065041584, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 247502 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:56865 is added to blk_1073741827_1003{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-151a7f34-1006-4de7-b6ea-e4935f23f6e5:NORMAL:127.0.0.1:43363|RBW], ReplicaUC[[DISK]DS-771019a8-18cc-4759-8154-7923750f1dfc:NORMAL:127.0.0.1:56865|FINALIZED]]} size 0
[junit4] 2> 247504 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:43363 is added to blk_1073741827_1003{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-771019a8-18cc-4759-8154-7923750f1dfc:NORMAL:127.0.0.1:56865|FINALIZED], ReplicaUC[[DISK]DS-6f1867bb-3c3b-49a1-8466-76f7bd52f547:NORMAL:127.0.0.1:43363|FINALIZED]]} size 0
[junit4] 2> 247520 WARN (coreLoadExecutor-342-thread-1-processing-n:127.0.0.1:57301_v%2Fe) [n:127.0.0.1:57301_v%2Fe c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 248012 INFO (coreLoadExecutor-342-thread-1-processing-n:127.0.0.1:57301_v%2Fe) [n:127.0.0.1:57301_v%2Fe c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 248012 INFO (coreLoadExecutor-342-thread-1-processing-n:127.0.0.1:57301_v%2Fe) [n:127.0.0.1:57301_v%2Fe c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 248012 INFO (coreLoadExecutor-342-thread-1-processing-n:127.0.0.1:57301_v%2Fe) [n:127.0.0.1:57301_v%2Fe c:collection1 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 248064 INFO (coreLoadExecutor-342-thread-1-processing-n:127.0.0.1:57301_v%2Fe) [n:127.0.0.1:57301_v%2Fe c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 248064 INFO (coreLoadExecutor-342-thread-1-processing-n:127.0.0.1:57301_v%2Fe) [n:127.0.0.1:57301_v%2Fe c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 248067 INFO (coreLoadExecutor-342-thread-1-processing-n:127.0.0.1:57301_v%2Fe) [n:127.0.0.1:57301_v%2Fe c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=33, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4500553319634587]
[junit4] 2> 248097 INFO (coreLoadExecutor-342-thread-1-processing-n:127.0.0.1:57301_v%2Fe) [n:127.0.0.1:57301_v%2Fe c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@62f89ae0[collection1] main]
[junit4] 2> 248098 INFO (coreLoadExecutor-342-thread-1-processing-n:127.0.0.1:57301_v%2Fe) [n:127.0.0.1:57301_v%2Fe c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 248099 INFO (coreLoadExecutor-342-thread-1-processing-n:127.0.0.1:57301_v%2Fe) [n:127.0.0.1:57301_v%2Fe c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 248099 INFO (coreLoadExecutor-342-thread-1-processing-n:127.0.0.1:57301_v%2Fe) [n:127.0.0.1:57301_v%2Fe c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 248100 INFO (searcherExecutor-343-thread-1-processing-n:127.0.0.1:57301_v%2Fe x:collection1 c:collection1) [n:127.0.0.1:57301_v%2Fe c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@62f89ae0[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 248119 INFO (coreLoadExecutor-342-thread-1-processing-n:127.0.0.1:57301_v%2Fe) [n:127.0.0.1:57301_v%2Fe c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577661061379653632
[junit4] 2> 248156 INFO (zkCallback-101-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 248159 INFO (zkCallback-107-thread-1-processing-n:127.0.0.1:57301_v%2Fe) [n:127.0.0.1:57301_v%2Fe ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 248160 INFO (coreZkRegister-337-thread-1-processing-n:127.0.0.1:57301_v%2Fe x:collection1 c:collection1) [n:127.0.0.1:57301_v%2Fe c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 248160 INFO (coreZkRegister-337-thread-1-processing-n:127.0.0.1:57301_v%2Fe x:collection1 c:collection1) [n:127.0.0.1:57301_v%2Fe c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 248160 INFO (coreZkRegister-337-thread-1-processing-n:127.0.0.1:57301_v%2Fe x:collection1 c:collection1) [n:127.0.0.1:57301_v%2Fe c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:57301/v/e/collection1/
[junit4] 2> 248160 INFO (coreZkRegister-337-thread-1-processing-n:127.0.0.1:57301_v%2Fe x:collection1 c:collection1) [n:127.0.0.1:57301_v%2Fe c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 248160 INFO (coreZkRegister-337-thread-1-processing-n:127.0.0.1:57301_v%2Fe x:collection1 c:collection1) [n:127.0.0.1:57301_v%2Fe c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:57301/v/e/collection1/ has no replicas
[junit4] 2> 248160 INFO (coreZkRegister-337-thread-1-processing-n:127.0.0.1:57301_v%2Fe x:collection1 c:collection1) [n:127.0.0.1:57301_v%2Fe c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 248543 INFO (coreZkRegister-337-thread-1-processing-n:127.0.0.1:57301_v%2Fe x:collection1 c:collection1) [n:127.0.0.1:57301_v%2Fe c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:57301/v/e/collection1/ shard1
[junit4] 2> 248605 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_8F84846AADE38943-001/shard-3-001/cores/collection1
[junit4] 2> 248606 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_8F84846AADE38943-001/shard-3-001
[junit4] 2> 248607 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 248616 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2fb2f2dc{/v/e,null,AVAILABLE}
[junit4] 2> 248616 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@36453b93{HTTP/1.1,[http/1.1]}{127.0.0.1:42307}
[junit4] 2> 248616 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.e.j.s.Server Started @256769ms
[junit4] 2> 248616 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml, hostContext=/v/e, hostPort=42307, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_8F84846AADE38943-001/shard-3-001/cores}
[junit4] 2> 248616 ERROR (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 248617 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.6.1
[junit4] 2> 248617 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 248617 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 248617 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-09-05T01:27:14.730Z
[junit4] 2> 248632 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 248632 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_8F84846AADE38943-001/shard-3-001/solr.xml
[junit4] 2> 248641 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 248641 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:59534/solr
[junit4] 2> 248646 INFO (zkCallback-107-thread-1-processing-n:127.0.0.1:57301_v%2Fe) [n:127.0.0.1:57301_v%2Fe ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 248646 INFO (zkCallback-101-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 248657 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [n:127.0.0.1:42307_v%2Fe ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 248658 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [n:127.0.0.1:42307_v%2Fe ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 248659 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [n:127.0.0.1:42307_v%2Fe ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42307_v%2Fe
[junit4] 2> 248660 INFO (zkCallback-107-thread-1-processing-n:127.0.0.1:57301_v%2Fe) [n:127.0.0.1:57301_v%2Fe ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 248660 INFO (zkCallback-95-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 248660 INFO (zkCallback-101-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 248660 INFO (zkCallback-91-thread-2-processing-n:127.0.0.1:58634_v%2Fe) [n:127.0.0.1:58634_v%2Fe ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 248672 INFO (zkCallback-113-thread-1-processing-n:127.0.0.1:42307_v%2Fe) [n:127.0.0.1:42307_v%2Fe ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 248694 INFO (coreZkRegister-337-thread-1-processing-n:127.0.0.1:57301_v%2Fe x:collection1 c:collection1) [n:127.0.0.1:57301_v%2Fe c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 248714 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [n:127.0.0.1:42307_v%2Fe ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_8F84846AADE38943-001/shard-3-001/cores
[junit4] 2> 248714 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[8F84846AADE38943]) [n:127.0.0.1:42307_v%2Fe ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 248726 INFO (coreLoadExecutor-353-thread-1-processing-n:127.0.0.1:42307_v%2Fe) [n:127.0.0.1:42307_v%2Fe c:collection1 x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 248727 INFO (OverseerStateUpdate-98603815251673092-127.0.0.1:58634_v%2Fe-n_0000000000) [n:127.0.0.1:58634_v%2Fe ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 248866 INFO (zkCallback-101-thread-1-processing-n:127.0.0.1:60477_v%2Fe) [n:127.0.0.1:60477_v%2Fe ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 248866 INFO (zkCallback-107-thread-1-processing-n:127.0.0.1:57301_v%2Fe) [n:127.0.0.1:57301_v%2Fe ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 248866 INFO (zkCallback-113-thread-1-processing-n:127.0.0.1:42307_v%2Fe) [n:127.0.0.1:42307_v%2Fe ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 249765 WARN (coreLoadExecutor-353-thread-1-processing-n:127.0.0.1:42307_v%2Fe) [n:127.0.0.1:42307_v%2Fe c:collection1 x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 249766 INFO (coreLoadExecutor-353-thread-1-processing-n:127.0.0.1:42307_v%2Fe) [n:127.0.0.1:42307_v%2Fe c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.1
[junit4] 2> 249819 INFO (coreLoadExecutor-353-thread-1-processing-n:127.0.0.1:42307_v%2Fe) [n:127.0.0.1:42307_v%2Fe c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 250208 INFO (coreLoadExecutor-353-thread-1-processing-n:127.0.0.1:42307_v%2Fe) [n:127.0.0.1:42307_v%2Fe c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 250268 INFO (coreLoadExecutor-353-thread-1-processing-n:127.0.0.1:42307_v%2Fe) [n:127.0.0.1:42307_v%2Fe c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1, trusted=true
[junit4] 2> 250271 INFO (coreLoadExecutor-353-thread-1-processing-n:127.0.0.1:42307_v%2Fe) [n:127.0.0.1:42307_v%2Fe c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:57357/solr_hdfs_home
[junit4] 2> 250272 INFO (coreLoadExecutor-353-thread-1-processing-n:127.0.0.1:42307_v%2Fe) [n:127.0.0.1:42307_v%2Fe c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 250272 INFO (coreLoadExecutor-353-thread-1-processing-n:127.0.0.1:42307_v%2Fe) [n:127.0.0.1:42307_v%2Fe c:collection1 x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 250272 INFO (coreLoadExecutor-353-thread-1-processing-n:127.0.0.1:42307_v%2Fe) [n:127.0.0.1:42307_v%2Fe c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_8F84846AADE38943-001/shard-3-001/cores/collection1], dataDir=[hdfs://localhost:57357/solr_hdfs_home/collection1/core_node3/data/]
[junit4] 2> 250272 INFO (coreLoadExecutor-353-thread-1-processing-n:127.0.0.1:42307_v%2Fe) [n:127.0.0.1:42307_v%2Fe c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@41c15ff1
[junit4] 2> 250272 INFO (coreLoadExecutor-353-thread-1-processing-n:127.0.0.1:42307_v%2Fe) [n:127.0.0.1:42307_v%2Fe c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:57357/solr_hdfs_home/collection1/core_node3/data/snapshot_metadata
[junit4] 2> 250353 INFO (coreLoadExecutor-353-thread-1-processing-n:127.0.0.1:42307_v%2Fe) [n:127.0.0.1:42307_v%2Fe c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 250353 INFO (coreLoadExecutor-353-thread-1-processing-n:127.0.0.1:42307_v%2Fe) [n:127.0.0.1:42
[...truncated too long message...]
OTSTRAP&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 509174 INFO (qtp548863479-2296) [n:127.0.0.1:39831_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica1] o.a.s.c.S.Request [cdcr-target_shard1_replica1] webapp=/solr path=/cdcr params={qt=/cdcr&action=BOOTSTRAP_STATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 509175 WARN (cdcr-bootstrap-status-431-thread-1-processing-n:127.0.0.1:52142_solr x:cdcr-source_shard1_replica1 s:shard1 c:cdcr-source r:core_node2) [n:127.0.0.1:52142_solr c:cdcr-source s:shard1 r:core_node2 x:cdcr-source_shard1_replica1] o.a.s.h.CdcrReplicatorManager Cancelling waiting for bootstrap on target: cdcr-target shard: shard1 to complete
[junit4] 2> 509273 INFO (coreCloseExecutor-936-thread-1) [n:127.0.0.1:52142_solr c:cdcr-source s:shard1 r:core_node2 x:cdcr-source_shard1_replica1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.cdcr-source.shard1.replica1
[junit4] 2> 509670 INFO (jetty-closer-406-thread-1) [ ] o.a.s.c.Overseer Overseer (id=98603823430631435-127.0.0.1:52142_solr-n_0000000001) closing
[junit4] 2> 509670 INFO (OverseerStateUpdate-98603823430631435-127.0.0.1:52142_solr-n_0000000001) [n:127.0.0.1:52142_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:52142_solr
[junit4] 2> 511182 WARN (zkCallback-425-thread-2-processing-n:127.0.0.1:52142_solr) [n:127.0.0.1:52142_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 511183 INFO (jetty-closer-406-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@704142b6{/solr,null,UNAVAILABLE}
[junit4] 2> 511183 ERROR (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[8F84846AADE38943]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 511184 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[8F84846AADE38943]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:47690 47690
[junit4] 2> 511316 INFO (Thread-486) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:47690 47690
[junit4] 2> 511316 WARN (Thread-486) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/aliases.json
[junit4] 2> 3 /solr/clusterprops.json
[junit4] 2> 2 /solr/configs/cdcr-source
[junit4] 2> 2 /solr/security.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/clusterstate.json
[junit4] 2> 2 /solr/collections/cdcr-source/cdcr/state/process
[junit4] 2> 2 /solr/collections/cdcr-source/state.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/collections
[junit4] 2> 2 /solr/overseer/queue
[junit4] 2> 2 /solr/live_nodes
[junit4] 2> 2 /solr/overseer/collection-queue-work
[junit4] 2> 2 /solr/overseer/queue-work
[junit4] 2>
[junit4] 2> 511335 INFO (jetty-closer-394-thread-1) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@604de7b6{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 511336 INFO (jetty-closer-394-thread-1) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1692596823
[junit4] 2> 511336 INFO (jetty-closer-394-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.node
[junit4] 2> 511336 INFO (jetty-closer-394-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.jvm
[junit4] 2> 511336 INFO (jetty-closer-394-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.jetty
[junit4] 2> 511355 INFO (coreCloseExecutor-938-thread-1) [n:127.0.0.1:39831_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica1] o.a.s.c.SolrCore [cdcr-target_shard1_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@3218c1c2
[junit4] 2> 511355 INFO (coreCloseExecutor-938-thread-1) [n:127.0.0.1:39831_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica1] o.a.s.h.CdcrRequestHandler Solr core is being closed - shutting down CDCR handler @ cdcr-target:shard1
[junit4] 2> 511362 INFO (zkCallback-400-thread-2-processing-n:127.0.0.1:39831_solr) [n:127.0.0.1:39831_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-target/state.json] for collection [cdcr-target] has occurred - updating... (live nodes size: [1])
[junit4] 2> 511415 INFO (coreCloseExecutor-938-thread-1) [n:127.0.0.1:39831_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.cdcr-target.shard1.replica1
[junit4] 2> 511416 INFO (jetty-closer-394-thread-1) [ ] o.a.s.c.Overseer Overseer (id=98603823361818627-127.0.0.1:39831_solr-n_0000000000) closing
[junit4] 2> 511416 INFO (OverseerStateUpdate-98603823361818627-127.0.0.1:39831_solr-n_0000000000) [n:127.0.0.1:39831_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:39831_solr
[junit4] 2> 511425 WARN (zkCallback-400-thread-3-processing-n:127.0.0.1:39831_solr) [n:127.0.0.1:39831_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 511430 INFO (jetty-closer-394-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@6d1e9bb8{/solr,null,UNAVAILABLE}
[junit4] 2> 511430 ERROR (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[8F84846AADE38943]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 511431 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[8F84846AADE38943]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:54217 54217
[junit4] 2> 511534 INFO (Thread-476) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:54217 54217
[junit4] 2> 511534 WARN (Thread-476) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/aliases.json
[junit4] 2> 3 /solr/clusterprops.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/clusterstate.json
[junit4] 2> 2 /solr/collections/cdcr-target/cdcr/state/process
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/live_nodes
[junit4] 2> 3 /solr/collections
[junit4] 2>
[junit4] 2> 511535 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[8F84846AADE38943]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testConvertClusterToCdcrAndBootstrap
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=CdcrBootstrapTest -Dtests.method=testConvertClusterToCdcrAndBootstrap -Dtests.seed=8F84846AADE38943 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/test-data/enwiki.random.lines.txt -Dtests.locale=en-SG -Dtests.timezone=IST -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] FAILURE 154s J1 | CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap <<<
[junit4] > Throwable #1: java.lang.AssertionError: Document mismatch on target after sync expected:<10000> but was:<0>
[junit4] > at __randomizedtesting.SeedInfo.seed([8F84846AADE38943:5853AB1D19BC1104]:0)
[junit4] > at org.apache.solr.cloud.CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap(CdcrBootstrapTest.java:134)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 511542 INFO (SUITE-CdcrBootstrapTest-seed#[8F84846AADE38943]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrBootstrapTest_8F84846AADE38943-001
[junit4] 2> Sep 05, 2017 1:31:35 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Lucene62, sim=RandomSimilarity(queryNorm=true,coord=crazy): {}, locale=en-SG, timezone=IST
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=211776360,total=459800576
[junit4] 2> NOTE: All tests run in this JVM: [DistributedDebugComponentTest, ZkCLITest, DistributedQueryComponentOptimizationTest, TestCSVResponseWriter, TestPartialUpdateDeduplication, BlockJoinFacetDistribTest, TestImpersonationWithHadoopAuth, TermVectorComponentTest, TestCharFilters, TestManagedStopFilterFactory, TestCollationFieldDocValues, TestManagedSchemaAPI, TestTestInjection, TestUseDocValuesAsStored2, CollectionsAPISolrJTest, MoreLikeThisHandlerTest, SaslZkACLProviderTest, TestExclusionRuleCollectionAccess, TestSolrIndexConfig, SpellCheckCollatorTest, TestFieldCollectionResource, TestSearchPerf, TestIntervalFaceting, TestConfigReload, LeaderInitiatedRecoveryOnShardRestartTest, TestShardHandlerFactory, IndexBasedSpellCheckerTest, TestCloudDeleteByQuery, CdcrBootstrapTest]
[junit4] Completed [68/713 (2!)] on J1 in 223.17s, 3 tests, 1 failure <<< FAILURES!
[...truncated 45607 lines...]
[JENKINS] Lucene-Solr-NightlyTests-6.6 - Build # 29 - Still
unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.6/29/
6 tests failed.
FAILED: org.apache.solr.cloud.ShardSplitTest.testSplitAfterFailedSplit
Error Message:
expected:<1> but was:<2>
Stack Trace:
java.lang.AssertionError: expected:<1> but was:<2>
at __randomizedtesting.SeedInfo.seed([7024D709DA7A4582:896944A6E60F0808]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.ShardSplitTest.testSplitAfterFailedSplit(ShardSplitTest.java:280)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testCollectionsAPI
Error Message:
Something is broken in the assert for no shards using the same indexDir - probably something was changed in the attributes published in the MBean of SolrCore : {}
Stack Trace:
java.lang.AssertionError: Something is broken in the assert for no shards using the same indexDir - probably something was changed in the attributes published in the MBean of SolrCore : {}
at __randomizedtesting.SeedInfo.seed([7024D709DA7A4582:3851A3BDDC496A17]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.checkNoTwoShardsUseTheSameIndexDir(CollectionsAPIDistributedZkTest.java:646)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:524)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.core.HdfsDirectoryFactoryTest.testInitArgsOrSysPropConfig
Error Message:
The max direct memory is likely too low. Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~134.217728 MB.
Stack Trace:
java.lang.RuntimeException: The max direct memory is likely too low. Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~134.217728 MB.
at __randomizedtesting.SeedInfo.seed([7024D709DA7A4582:878B1E2227F3AFA9]:0)
at org.apache.solr.core.HdfsDirectoryFactory.createBlockCache(HdfsDirectoryFactory.java:312)
at org.apache.solr.core.HdfsDirectoryFactory.getBlockDirectoryCache(HdfsDirectoryFactory.java:288)
at org.apache.solr.core.HdfsDirectoryFactory.create(HdfsDirectoryFactory.java:228)
at org.apache.solr.core.HdfsDirectoryFactoryTest.testInitArgsOrSysPropConfig(HdfsDirectoryFactoryTest.java:108)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Bits.java:694)
at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
at org.apache.solr.store.blockcache.BlockCache.<init>(BlockCache.java:75)
at org.apache.solr.core.HdfsDirectoryFactory.createBlockCache(HdfsDirectoryFactory.java:310)
... 42 more
FAILED: org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([823FD0F99C02D107]:0)
FAILED: junit.framework.TestSuite.org.apache.lucene.spatial3d.TestGeo3DPoint
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([823FD0F99C02D107]:0)
FAILED: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test
Error Message:
Captured an uncaught exception in thread: Thread[id=19471, name=Thread-8585, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=19471, name=Thread-8585, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Caused by: org.apache.solr.common.SolrException: Could not load collection from ZK: collection2
at __randomizedtesting.SeedInfo.seed([7024D709DA7A4582]:0)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1098)
at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:638)
at org.apache.solr.client.solrj.impl.CloudSolrClient.getDocCollection(CloudSolrClient.java:1489)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1099)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1073)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:177)
at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest$1IndexThread.run(FullSolrCloudDistribCmdsTest.java:641)
Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection2/state.json
at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1212)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:357)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:354)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:354)
at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1110)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1096)
... 7 more
Build Log:
[...truncated 10332 lines...]
[junit4] Suite: org.apache.lucene.spatial3d.TestGeo3DPoint
[junit4] 2> szept. 03, 2017 12:25:21 DE com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
[junit4] 2> WARNING: Suite execution timed out: org.apache.lucene.spatial3d.TestGeo3DPoint
[junit4] 2> 1) Thread[id=9, name=JUnit4-serializer-daemon, state=TIMED_WAITING, group=main]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:50)
[junit4] 2> 2) Thread[id=1, name=main, state=WAITING, group=main]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Thread.join(Thread.java:1252)
[junit4] 2> at java.lang.Thread.join(Thread.java:1326)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:608)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:457)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:244)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:355)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:13)
[junit4] 2> 3) Thread[id=13, name=SUITE-TestGeo3DPoint-seed#[823FD0F99C02D107], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at java.lang.Thread.getStackTrace(Thread.java:1559)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:696)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:693)
[junit4] 2> at java.security.AccessController.doPrivileged(Native Method)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getStackTrace(ThreadLeakControl.java:693)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getThreadsWithTraces(ThreadLeakControl.java:709)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:689)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:65)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:415)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:678)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:140)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:598)
[junit4] 2> 4) Thread[id=14, name=TEST-TestGeo3DPoint.testRandomBig-seed#[823FD0F99C02D107], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoCompositeMembershipShape.isWithin(GeoCompositeMembershipShape.java:54)
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoCompositeMembershipShape.isWithin(GeoCompositeMembershipShape.java:46)
[junit4] 2> at org.apache.lucene.spatial3d.geom.BaseXYZSolid.isAreaInsideShape(BaseXYZSolid.java:130)
[junit4] 2> at org.apache.lucene.spatial3d.geom.StandardXYZSolid.getRelationship(StandardXYZSolid.java:383)
[junit4] 2> at org.apache.lucene.spatial3d.PointInShapeIntersectVisitor.compare(PointInShapeIntersectVisitor.java:107)
[junit4] 2> at org.apache.lucene.codecs.asserting.AssertingPointsFormat$AssertingIntersectVisitor.compare(AssertingPointsFormat.java:147)
[junit4] 2> at org.apache.lucene.util.bkd.BKDReader.intersect(BKDReader.java:692)
[junit4] 2> at org.apache.lucene.util.bkd.BKDReader.intersect(BKDReader.java:743)
[junit4] 2> at org.apache.lucene.util.bkd.BKDReader.intersect(BKDReader.java:733)
[junit4] 2> at org.apache.lucene.util.bkd.BKDReader.intersect(BKDReader.java:743)
[junit4] 2> at org.apache.lucene.util.bkd.BKDReader.intersect(BKDReader.java:733)
[junit4] 2> at org.apache.lucene.util.bkd.BKDReader.intersect(BKDReader.java:733)
[junit4] 2> at org.apache.lucene.util.bkd.BKDReader.intersect(BKDReader.java:743)
[junit4] 2> at org.apache.lucene.util.bkd.BKDReader.intersect(BKDReader.java:743)
[junit4] 2> at org.apache.lucene.util.bkd.BKDReader.intersect(BKDReader.java:743)
[junit4] 2> at org.apache.lucene.util.bkd.BKDReader.intersect(BKDReader.java:743)
[junit4] 2> at org.apache.lucene.util.bkd.BKDReader.intersect(BKDReader.java:522)
[junit4] 2> at org.apache.lucene.codecs.lucene60.Lucene60PointsReader.intersect(Lucene60PointsReader.java:142)
[junit4] 2> at org.apache.lucene.codecs.asserting.AssertingPointsFormat$AssertingPointsReader.intersect(AssertingPointsFormat.java:173)
[junit4] 2> at org.apache.lucene.spatial3d.PointInGeo3DShapeQuery$1.scorer(PointInGeo3DShapeQuery.java:102)
[junit4] 2> at org.apache.lucene.search.Weight.bulkScorer(Weight.java:160)
[junit4] 2> at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.cache(LRUQueryCache.java:704)
[junit4] 2> at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.bulkScorer(LRUQueryCache.java:807)
[junit4] 2> at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:79)
[junit4] 2> at org.apache.lucene.search.AssertingWeight.bulkScorer(AssertingWeight.java:79)
[junit4] 2> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:665)
[junit4] 2> at org.apache.lucene.search.AssertingIndexSearcher.search(AssertingIndexSearcher.java:91)
[junit4] 2> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:472)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.verify(TestGeo3DPoint.java:814)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.doTestRandom(TestGeo3DPoint.java:548)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig(TestGeo3DPoint.java:480)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestGeo3DPoint -Dtests.method=testRandomBig -Dtests.seed=823FD0F99C02D107 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/test-data/enwiki.random.lines.txt -Dtests.locale=hu -Dtests.timezone=America/Costa_Rica -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 6736s J1 | TestGeo3DPoint.testRandomBig <<<
[junit4] > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
[junit4] > at __randomizedtesting.SeedInfo.seed([823FD0F99C02D107]:0)
[junit4] 2> szept. 03, 2017 12:25:21 DE com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> szept. 03, 2017 12:25:42 DE com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.lucene.spatial3d.TestGeo3DPoint:
[junit4] 2> 1) Thread[id=14, name=TEST-TestGeo3DPoint.testRandomBig-seed#[823FD0F99C02D107], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at org.apache.lucene.spatial3d.geom.Plane.findIntersections(Plane.java:800)
[junit4] 2> at org.apache.lucene.spatial3d.geom.Plane.recordBounds(Plane.java:1251)
[junit4] 2> at org.apache.lucene.spatial3d.geom.XYZBounds.addPlane(XYZBounds.java:161)
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoConvexPolygon.getBounds(GeoConvexPolygon.java:423)
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoCompositeMembershipShape.getBounds(GeoCompositeMembershipShape.java:75)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.verify(TestGeo3DPoint.java:845)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.doTestRandom(TestGeo3DPoint.java:548)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig(TestGeo3DPoint.java:480)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> szept. 03, 2017 12:25:42 DE com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=14, name=TEST-TestGeo3DPoint.testRandomBig-seed#[823FD0F99C02D107], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> szept. 03, 2017 12:25:45 DE com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> SEVERE: There are still zombie threads that couldn't be terminated:
[junit4] 2> 1) Thread[id=14, name=TEST-TestGeo3DPoint.testRandomBig-seed#[823FD0F99C02D107], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at org.apache.lucene.spatial3d.geom.Plane.recordBounds(Plane.java:1409)
[junit4] 2> at org.apache.lucene.spatial3d.geom.XYZBounds.addPlane(XYZBounds.java:161)
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoConvexPolygon.getBounds(GeoConvexPolygon.java:423)
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoCompositeMembershipShape.getBounds(GeoCompositeMembershipShape.java:75)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.verify(TestGeo3DPoint.java:845)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.doTestRandom(TestGeo3DPoint.java:548)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig(TestGeo3DPoint.java:480)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): {id=PostingsFormat(name=MockRandom)}, docValues:{id=DocValuesFormat(name=Direct), point=DocValuesFormat(name=Asserting)}, maxPointsInLeafNode=116, maxMBSortInHeap=7.283981539355439, sim=RandomSimilarity(queryNorm=false,coord=yes): {}, locale=hu, timezone=America/Costa_Rica
[junit4] 2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=2,free=140775120,total=410517504
[junit4] 2> NOTE: All tests run in this JVM: [GeoPointTest, TestGeo3DPoint]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestGeo3DPoint -Dtests.seed=823FD0F99C02D107 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/test-data/enwiki.random.lines.txt -Dtests.locale=hu -Dtests.timezone=America/Costa_Rica -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J1 | TestGeo3DPoint (suite) <<<
[junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4] > at __randomizedtesting.SeedInfo.seed([823FD0F99C02D107]:0)
[junit4] Completed [11/11 (1!)] on J1 in 7223.72s, 6 tests, 2 errors <<< FAILURES!
[...truncated 2370 lines...]
[junit4] Suite: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_7024D709DA7A4582-001/init-core-data-001
[junit4] 2> 3787903 WARN (SUITE-FullSolrCloudDistribCmdsTest-seed#[7024D709DA7A4582]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=2 numCloses=2
[junit4] 2> 3787903 INFO (SUITE-FullSolrCloudDistribCmdsTest-seed#[7024D709DA7A4582]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields
[junit4] 2> 3787904 INFO (SUITE-FullSolrCloudDistribCmdsTest-seed#[7024D709DA7A4582]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
[junit4] 2> 3787904 INFO (SUITE-FullSolrCloudDistribCmdsTest-seed#[7024D709DA7A4582]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /cn_jp/y
[junit4] 2> 3787934 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 3787962 INFO (Thread-8489) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 3787962 INFO (Thread-8489) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 3787978 ERROR (Thread-8489) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 3788063 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.c.ZkTestServer start zk server on port:44967
[junit4] 2> 3788146 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 3788147 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 3788148 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 3788161 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 3788163 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 3788163 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 3788164 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 3788164 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 3788181 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 3788226 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 3788227 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 3788227 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e46acc9ba0001, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 3788735 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_7024D709DA7A4582-001/control-001/cores/collection1
[junit4] 2> 3788752 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 3788847 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@32f90a88{/cn_jp/y,null,AVAILABLE}
[junit4] 2> 3788848 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@165cfe50{HTTP/1.1,[http/1.1]}{127.0.0.1:38021}
[junit4] 2> 3788848 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.e.j.s.Server Started @3800390ms
[junit4] 2> 3788848 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_7024D709DA7A4582-001/tempDir-001/control/data, hostContext=/cn_jp/y, hostPort=38021, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_7024D709DA7A4582-001/control-001/cores}
[junit4] 2> 3788848 ERROR (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3788848 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.6.1
[junit4] 2> 3788848 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3788848 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3788848 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-09-03T07:38:03.440Z
[junit4] 2> 3788897 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 3788897 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_7024D709DA7A4582-001/control-001/solr.xml
[junit4] 2> 3788931 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 3788945 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44967/solr
[junit4] 2> 3789494 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [n:127.0.0.1:38021_cn_jp%2Fy ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3789494 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [n:127.0.0.1:38021_cn_jp%2Fy ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:38021_cn_jp%2Fy
[junit4] 2> 3789495 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [n:127.0.0.1:38021_cn_jp%2Fy ] o.a.s.c.Overseer Overseer (id=98593949781458948-127.0.0.1:38021_cn_jp%2Fy-n_0000000000) starting
[junit4] 2> 3789611 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [n:127.0.0.1:38021_cn_jp%2Fy ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38021_cn_jp%2Fy
[junit4] 2> 3789613 INFO (zkCallback-2282-thread-1-processing-n:127.0.0.1:38021_cn_jp%2Fy) [n:127.0.0.1:38021_cn_jp%2Fy ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3790410 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [n:127.0.0.1:38021_cn_jp%2Fy ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_7024D709DA7A4582-001/control-001/cores
[junit4] 2> 3790410 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [n:127.0.0.1:38021_cn_jp%2Fy ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 3790439 INFO (coreLoadExecutor-4894-thread-1-processing-n:127.0.0.1:38021_cn_jp%2Fy) [n:127.0.0.1:38021_cn_jp%2Fy c:control_collection x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 3790538 INFO (OverseerStateUpdate-98593949781458948-127.0.0.1:38021_cn_jp%2Fy-n_0000000000) [n:127.0.0.1:38021_cn_jp%2Fy ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 3791578 WARN (coreLoadExecutor-4894-thread-1-processing-n:127.0.0.1:38021_cn_jp%2Fy) [n:127.0.0.1:38021_cn_jp%2Fy c:control_collection x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 3791579 INFO (coreLoadExecutor-4894-thread-1-processing-n:127.0.0.1:38021_cn_jp%2Fy) [n:127.0.0.1:38021_cn_jp%2Fy c:control_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.1
[junit4] 2> 3791637 INFO (coreLoadExecutor-4894-thread-1-processing-n:127.0.0.1:38021_cn_jp%2Fy) [n:127.0.0.1:38021_cn_jp%2Fy c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 3791999 INFO (coreLoadExecutor-4894-thread-1-processing-n:127.0.0.1:38021_cn_jp%2Fy) [n:127.0.0.1:38021_cn_jp%2Fy c:control_collection x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 3792141 INFO (coreLoadExecutor-4894-thread-1-processing-n:127.0.0.1:38021_cn_jp%2Fy) [n:127.0.0.1:38021_cn_jp%2Fy c:control_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection, trusted=true
[junit4] 2> 3792142 INFO (coreLoadExecutor-4894-thread-1-processing-n:127.0.0.1:38021_cn_jp%2Fy) [n:127.0.0.1:38021_cn_jp%2Fy c:control_collection x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 3792142 INFO (coreLoadExecutor-4894-thread-1-processing-n:127.0.0.1:38021_cn_jp%2Fy) [n:127.0.0.1:38021_cn_jp%2Fy c:control_collection x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_7024D709DA7A4582-001/control-001/cores/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_7024D709DA7A4582-001/control-001/cores/collection1/data/]
[junit4] 2> 3792142 INFO (coreLoadExecutor-4894-thread-1-processing-n:127.0.0.1:38021_cn_jp%2Fy) [n:127.0.0.1:38021_cn_jp%2Fy c:control_collection x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4377959a
[junit4] 2> 3792143 INFO (coreLoadExecutor-4894-thread-1-processing-n:127.0.0.1:38021_cn_jp%2Fy) [n:127.0.0.1:38021_cn_jp%2Fy c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=46, maxMergeAtOnceExplicit=21, maxMergedSegmentMB=72.4404296875, floorSegmentMB=1.3857421875, forceMergeDeletesPctAllowed=6.271570584226856, segmentsPerTier=25.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 3792157 WARN (coreLoadExecutor-4894-thread-1-processing-n:127.0.0.1:38021_cn_jp%2Fy) [n:127.0.0.1:38021_cn_jp%2Fy c:control_collection x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 3792330 INFO (coreLoadExecutor-4894-thread-1-processing-n:127.0.0.1:38021_cn_jp%2Fy) [n:127.0.0.1:38021_cn_jp%2Fy c:control_collection x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 3792330 INFO (coreLoadExecutor-4894-thread-1-processing-n:127.0.0.1:38021_cn_jp%2Fy) [n:127.0.0.1:38021_cn_jp%2Fy c:control_collection x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 3792331 INFO (coreLoadExecutor-4894-thread-1-processing-n:127.0.0.1:38021_cn_jp%2Fy) [n:127.0.0.1:38021_cn_jp%2Fy c:control_collection x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 3792331 INFO (coreLoadExecutor-4894-thread-1-processing-n:127.0.0.1:38021_cn_jp%2Fy) [n:127.0.0.1:38021_cn_jp%2Fy c:control_collection x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 3792332 INFO (coreLoadExecutor-4894-thread-1-processing-n:127.0.0.1:38021_cn_jp%2Fy) [n:127.0.0.1:38021_cn_jp%2Fy c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=49, maxMergeAtOnceExplicit=50, maxMergedSegmentMB=87.404296875, floorSegmentMB=0.640625, forceMergeDeletesPctAllowed=8.904845903998948, segmentsPerTier=14.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5417428287765911
[junit4] 2> 3792332 INFO (coreLoadExecutor-4894-thread-1-processing-n:127.0.0.1:38021_cn_jp%2Fy) [n:127.0.0.1:38021_cn_jp%2Fy c:control_collection x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4b947af1[collection1] main]
[junit4] 2> 3792332 INFO (coreLoadExecutor-4894-thread-1-processing-n:127.0.0.1:38021_cn_jp%2Fy) [n:127.0.0.1:38021_cn_jp%2Fy c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 3792333 INFO (coreLoadExecutor-4894-thread-1-processing-n:127.0.0.1:38021_cn_jp%2Fy) [n:127.0.0.1:38021_cn_jp%2Fy c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 3792333 INFO (coreLoadExecutor-4894-thread-1-processing-n:127.0.0.1:38021_cn_jp%2Fy) [n:127.0.0.1:38021_cn_jp%2Fy c:control_collection x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 3792347 INFO (searcherExecutor-4895-thread-1-processing-n:127.0.0.1:38021_cn_jp%2Fy x:collection1 c:control_collection) [n:127.0.0.1:38021_cn_jp%2Fy c:control_collection x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@4b947af1[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 3792348 INFO (coreLoadExecutor-4894-thread-1-processing-n:127.0.0.1:38021_cn_jp%2Fy) [n:127.0.0.1:38021_cn_jp%2Fy c:control_collection x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577503201102397440
[junit4] 2> 3792395 INFO (coreZkRegister-4887-thread-1-processing-n:127.0.0.1:38021_cn_jp%2Fy x:collection1 c:control_collection) [n:127.0.0.1:38021_cn_jp%2Fy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 3792395 INFO (coreZkRegister-4887-thread-1-processing-n:127.0.0.1:38021_cn_jp%2Fy x:collection1 c:control_collection) [n:127.0.0.1:38021_cn_jp%2Fy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 3792395 INFO (coreZkRegister-4887-thread-1-processing-n:127.0.0.1:38021_cn_jp%2Fy x:collection1 c:control_collection) [n:127.0.0.1:38021_cn_jp%2Fy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:38021/cn_jp/y/collection1/
[junit4] 2> 3792395 INFO (coreZkRegister-4887-thread-1-processing-n:127.0.0.1:38021_cn_jp%2Fy x:collection1 c:control_collection) [n:127.0.0.1:38021_cn_jp%2Fy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 3792395 INFO (coreZkRegister-4887-thread-1-processing-n:127.0.0.1:38021_cn_jp%2Fy x:collection1 c:control_collection) [n:127.0.0.1:38021_cn_jp%2Fy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:38021/cn_jp/y/collection1/ has no replicas
[junit4] 2> 3792395 INFO (coreZkRegister-4887-thread-1-processing-n:127.0.0.1:38021_cn_jp%2Fy x:collection1 c:control_collection) [n:127.0.0.1:38021_cn_jp%2Fy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 3792397 INFO (coreZkRegister-4887-thread-1-processing-n:127.0.0.1:38021_cn_jp%2Fy x:collection1 c:control_collection) [n:127.0.0.1:38021_cn_jp%2Fy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:38021/cn_jp/y/collection1/ shard1
[junit4] 2> 3792564 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3792565 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:44967/solr ready
[junit4] 2> 3792577 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 3792577 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
[junit4] 2> 3792601 INFO (coreZkRegister-4887-thread-1-processing-n:127.0.0.1:38021_cn_jp%2Fy x:collection1 c:control_collection) [n:127.0.0.1:38021_cn_jp%2Fy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 3792913 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_7024D709DA7A4582-001/shard-1-001/cores/collection1
[junit4] 2> 3792914 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_7024D709DA7A4582-001/shard-1-001
[junit4] 2> 3792914 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 3792990 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1efa0d9f{/cn_jp/y,null,AVAILABLE}
[junit4] 2> 3792991 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@5306b4e9{HTTP/1.1,[http/1.1]}{127.0.0.1:42003}
[junit4] 2> 3792991 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.e.j.s.Server Started @3804533ms
[junit4] 2> 3792991 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_7024D709DA7A4582-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/cn_jp/y, hostPort=42003, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_7024D709DA7A4582-001/shard-1-001/cores}
[junit4] 2> 3792991 ERROR (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3793019 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.6.1
[junit4] 2> 3793019 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3793019 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3793020 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-09-03T07:38:07.612Z
[junit4] 2> 3793130 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 3793130 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_7024D709DA7A4582-001/shard-1-001/solr.xml
[junit4] 2> 3793164 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 3793164 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44967/solr
[junit4] 2> 3793219 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [n:127.0.0.1:42003_cn_jp%2Fy ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3793313 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [n:127.0.0.1:42003_cn_jp%2Fy ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3793314 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [n:127.0.0.1:42003_cn_jp%2Fy ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42003_cn_jp%2Fy
[junit4] 2> 3793377 INFO (zkCallback-2292-thread-1-processing-n:127.0.0.1:42003_cn_jp%2Fy) [n:127.0.0.1:42003_cn_jp%2Fy ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 3793394 INFO (zkCallback-2286-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 3793393 INFO (zkCallback-2282-thread-1-processing-n:127.0.0.1:38021_cn_jp%2Fy) [n:127.0.0.1:38021_cn_jp%2Fy ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 3793698 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [n:127.0.0.1:42003_cn_jp%2Fy ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_7024D709DA7A4582-001/shard-1-001/cores
[junit4] 2> 3793699 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [n:127.0.0.1:42003_cn_jp%2Fy ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 3793701 INFO (coreLoadExecutor-4905-thread-1-processing-n:127.0.0.1:42003_cn_jp%2Fy) [n:127.0.0.1:42003_cn_jp%2Fy c:collection1 x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 3793768 INFO (OverseerStateUpdate-98593949781458948-127.0.0.1:38021_cn_jp%2Fy-n_0000000000) [n:127.0.0.1:38021_cn_jp%2Fy ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 3793984 INFO (zkCallback-2292-thread-1-processing-n:127.0.0.1:42003_cn_jp%2Fy) [n:127.0.0.1:42003_cn_jp%2Fy ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 3794823 WARN (coreLoadExecutor-4905-thread-1-processing-n:127.0.0.1:42003_cn_jp%2Fy) [n:127.0.0.1:42003_cn_jp%2Fy c:collection1 x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 3794823 INFO (coreLoadExecutor-4905-thread-1-processing-n:127.0.0.1:42003_cn_jp%2Fy) [n:127.0.0.1:42003_cn_jp%2Fy c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.1
[junit4] 2> 3794870 INFO (coreLoadExecutor-4905-thread-1-processing-n:127.0.0.1:42003_cn_jp%2Fy) [n:127.0.0.1:42003_cn_jp%2Fy c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 3795114 INFO (coreLoadExecutor-4905-thread-1-processing-n:127.0.0.1:42003_cn_jp%2Fy) [n:127.0.0.1:42003_cn_jp%2Fy c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 3795152 INFO (coreLoadExecutor-4905-thread-1-processing-n:127.0.0.1:42003_cn_jp%2Fy) [n:127.0.0.1:42003_cn_jp%2Fy c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1, trusted=true
[junit4] 2> 3795152 INFO (coreLoadExecutor-4905-thread-1-processing-n:127.0.0.1:42003_cn_jp%2Fy) [n:127.0.0.1:42003_cn_jp%2Fy c:collection1 x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 3795153 INFO (coreLoadExecutor-4905-thread-1-processing-n:127.0.0.1:42003_cn_jp%2Fy) [n:127.0.0.1:42003_cn_jp%2Fy c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_7024D709DA7A4582-001/shard-1-001/cores/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_7024D709DA7A4582-001/shard-1-001/cores/collection1/data/]
[junit4] 2> 3795153 INFO (coreLoadExecutor-4905-thread-1-processing-n:127.0.0.1:42003_cn_jp%2Fy) [n:127.0.0.1:42003_cn_jp%2Fy c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4377959a
[junit4] 2> 3795166 INFO (coreLoadExecutor-4905-thread-1-processing-n:127.0.0.1:42003_cn_jp%2Fy) [n:127.0.0.1:42003_cn_jp%2Fy c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=46, maxMergeAtOnceExplicit=21, maxMergedSegmentMB=72.4404296875, floorSegmentMB=1.3857421875, forceMergeDeletesPctAllowed=6.271570584226856, segmentsPerTier=25.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 3795286 WARN (coreLoadExecutor-4905-thread-1-processing-n:127.0.0.1:42003_cn_jp%2Fy) [n:127.0.0.1:42003_cn_jp%2Fy c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 3795415 INFO (coreLoadExecutor-4905-thread-1-processing-n:127.0.0.1:42003_cn_jp%2Fy) [n:127.0.0.1:42003_cn_jp%2Fy c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 3795415 INFO (coreLoadExecutor-4905-thread-1-processing-n:127.0.0.1:42003_cn_jp%2Fy) [n:127.0.0.1:42003_cn_jp%2Fy c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 3795416 INFO (coreLoadExecutor-4905-thread-1-processing-n:127.0.0.1:42003_cn_jp%2Fy) [n:127.0.0.1:42003_cn_jp%2Fy c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 3795416 INFO (coreLoadExecutor-4905-thread-1-processing-n:127.0.0.1:42003_cn_jp%2Fy) [n:127.0.0.1:42003_cn_jp%2Fy c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 3795417 INFO (coreLoadExecutor-4905-thread-1-processing-n:127.0.0.1:42003_cn_jp%2Fy) [n:127.0.0.1:42003_cn_jp%2Fy c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=49, maxMergeAtOnceExplicit=50, maxMergedSegmentMB=87.404296875, floorSegmentMB=0.640625, forceMergeDeletesPctAllowed=8.904845903998948, segmentsPerTier=14.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5417428287765911
[junit4] 2> 3795426 INFO (coreLoadExecutor-4905-thread-1-processing-n:127.0.0.1:42003_cn_jp%2Fy) [n:127.0.0.1:42003_cn_jp%2Fy c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4a94a8bf[collection1] main]
[junit4] 2> 3795427 INFO (coreLoadExecutor-4905-thread-1-processing-n:127.0.0.1:42003_cn_jp%2Fy) [n:127.0.0.1:42003_cn_jp%2Fy c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 3795427 INFO (coreLoadExecutor-4905-thread-1-processing-n:127.0.0.1:42003_cn_jp%2Fy) [n:127.0.0.1:42003_cn_jp%2Fy c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 3795427 INFO (coreLoadExecutor-4905-thread-1-processing-n:127.0.0.1:42003_cn_jp%2Fy) [n:127.0.0.1:42003_cn_jp%2Fy c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 3795429 INFO (searcherExecutor-4906-thread-1-processing-n:127.0.0.1:42003_cn_jp%2Fy x:collection1 c:collection1) [n:127.0.0.1:42003_cn_jp%2Fy c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@4a94a8bf[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 3795429 INFO (coreLoadExecutor-4905-thread-1-processing-n:127.0.0.1:42003_cn_jp%2Fy) [n:127.0.0.1:42003_cn_jp%2Fy c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577503204333060096
[junit4] 2> 3795460 INFO (coreZkRegister-4900-thread-1-processing-n:127.0.0.1:42003_cn_jp%2Fy x:collection1 c:collection1) [n:127.0.0.1:42003_cn_jp%2Fy c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 3795460 INFO (coreZkRegister-4900-thread-1-processing-n:127.0.0.1:42003_cn_jp%2Fy x:collection1 c:collection1) [n:127.0.0.1:42003_cn_jp%2Fy c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 3795460 INFO (coreZkRegister-4900-thread-1-processing-n:127.0.0.1:42003_cn_jp%2Fy x:collection1 c:collection1) [n:127.0.0.1:42003_cn_jp%2Fy c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:42003/cn_jp/y/collection1/
[junit4] 2> 3795460 INFO (coreZkRegister-4900-thread-1-processing-n:127.0.0.1:42003_cn_jp%2Fy x:collection1 c:collection1) [n:127.0.0.1:42003_cn_jp%2Fy c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 3795460 INFO (coreZkRegister-4900-thread-1-processing-n:127.0.0.1:42003_cn_jp%2Fy x:collection1 c:collection1) [n:127.0.0.1:42003_cn_jp%2Fy c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:42003/cn_jp/y/collection1/ has no replicas
[junit4] 2> 3795460 INFO (coreZkRegister-4900-thread-1-processing-n:127.0.0.1:42003_cn_jp%2Fy x:collection1 c:collection1) [n:127.0.0.1:42003_cn_jp%2Fy c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 3795475 INFO (coreZkRegister-4900-thread-1-processing-n:127.0.0.1:42003_cn_jp%2Fy x:collection1 c:collection1) [n:127.0.0.1:42003_cn_jp%2Fy c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:42003/cn_jp/y/collection1/ shard2
[junit4] 2> 3795588 INFO (zkCallback-2292-thread-1-processing-n:127.0.0.1:42003_cn_jp%2Fy) [n:127.0.0.1:42003_cn_jp%2Fy ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 3795639 INFO (coreZkRegister-4900-thread-1-processing-n:127.0.0.1:42003_cn_jp%2Fy x:collection1 c:collection1) [n:127.0.0.1:42003_cn_jp%2Fy c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 3795748 INFO (zkCallback-2292-thread-1-processing-n:127.0.0.1:42003_cn_jp%2Fy) [n:127.0.0.1:42003_cn_jp%2Fy ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 3796132 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_7024D709DA7A4582-001/shard-2-001/cores/collection1
[junit4] 2> 3796132 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_7024D709DA7A4582-001/shard-2-001
[junit4] 2> 3796133 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 3796146 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1f3ead29{/cn_jp/y,null,AVAILABLE}
[junit4] 2> 3796146 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1f93848{HTTP/1.1,[http/1.1]}{127.0.0.1:43274}
[junit4] 2> 3796146 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.e.j.s.Server Started @3807689ms
[junit4] 2> 3796146 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_7024D709DA7A4582-001/tempDir-001/jetty2, solrconfig=solrconfig.xml, hostContext=/cn_jp/y, hostPort=43274, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_7024D709DA7A4582-001/shard-2-001/cores}
[junit4] 2> 3796146 ERROR (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3796147 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.6.1
[junit4] 2> 3796147 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3796147 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3796147 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-09-03T07:38:10.739Z
[junit4] 2> 3796149 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 3796149 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_7024D709DA7A4582-001/shard-2-001/solr.xml
[junit4] 2> 3796179 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 3796180 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44967/solr
[junit4] 2> 3796196 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [n:127.0.0.1:43274_cn_jp%2Fy ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 3796197 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [n:127.0.0.1:43274_cn_jp%2Fy ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3796198 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [n:127.0.0.1:43274_cn_jp%2Fy ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43274_cn_jp%2Fy
[junit4] 2> 3796214 INFO (zkCallback-2298-thread-1-processing-n:127.0.0.1:43274_cn_jp%2Fy) [n:127.0.0.1:43274_cn_jp%2Fy ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 3796215 INFO (zkCallback-2292-thread-1-processing-n:127.0.0.1:42003_cn_jp%2Fy) [n:127.0.0.1:42003_cn_jp%2Fy ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 3796215 INFO (zkCallback-2282-thread-1-processing-n:127.0.0.1:38021_cn_jp%2Fy) [n:127.0.0.1:38021_cn_jp%2Fy ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 3796219 INFO (zkCallback-2286-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 3796461 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [n:127.0.0.1:43274_cn_jp%2Fy ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_7024D709DA7A4582-001/shard-2-001/cores
[junit4] 2> 3796461 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[7024D709DA7A4582]) [n:127.0.0.1:43274_cn_jp%2Fy ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 3796475 INFO (coreLoadExecutor-4916-thread-1-processing-n:127.0.0.1:43274_cn_jp%2Fy) [n:127.0.0.1:43274_cn_jp%2Fy c:collection1 x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 3796483 INFO (OverseerStateUpdate-98593949781458948-127.0.0.1:38021_cn_jp%2Fy-n_0000000000) [n:127.0.0.1:38021_cn_jp%2Fy ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard3
[junit4] 2> 3796589 INFO (zkCallback-2292-thread-1-processing-n:127.0.0.1:42003_cn_jp%2Fy) [n:127.0.0.1:42003_cn_jp%2Fy ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 3796589 INFO (zkCallback-2298-thread-1-processing-n:127.0.0.1:43274_cn_jp%2Fy) [n:127.0.0.1:43274_cn_jp%2Fy ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 3797567 WARN (coreLoadExecutor-4916-thread-1-processing-n:127.0.0.1:43274_cn_jp%2Fy) [n:127.0.0.1:43274_cn_jp%2Fy c:collection1 x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 3797568 INFO (coreLoadExecutor-4916-thread-1-processing-n:127.0.0.1:43274_cn_jp%2Fy) [n:127.0.0.1:43274_cn_jp%2Fy c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.1
[junit4] 2> 3797615 INFO (coreLoadExecutor-4916-thread-1-processing-n:127.0.0.1:43274_cn_jp%2Fy) [n:127.0.0.1:43274_cn_jp%2Fy c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 3797882 INFO (coreLoadExecutor-4916-thread-1-processing-n:127.0.0.1:43274_cn_jp%2Fy) [n:127.0.0.1:43274_cn_jp%2Fy c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 3797927 INFO (coreLoadExecutor-4916-thread-1-processing-n:127.0.0.1:43274_cn_jp%2Fy) [n:127.0.0.1:43274_cn_jp%2Fy c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1, trusted=true
[junit4] 2> 3797927 INFO (coreLoadExecutor-4916-thread-1-processing-n:127.0.0.1:43274_cn_jp%2Fy) [n:127.0.0.1:43274_cn_jp%2Fy c:collection1 x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 3797927 INFO (coreLoadExecutor-4916-thread-1-processing-n:127.0.0.1:43274_cn_jp%2Fy) [n:127.0.0.1:43274_cn_jp%2Fy c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_7024D709DA7A4582-001/shard-2-001/cores/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_7024D709DA7A4582-001/shard-2-001/cores/collection1/data/]
[junit4] 2> 3797928 INFO (coreLoadExecutor-4916-thread-1-processing-n:127.0.0.1:43274_cn_jp%2Fy) [n:127.0.0.1:43274_cn_jp%2Fy c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4377959a
[junit4] 2> 3797928 INFO (coreLoadExecutor-4916-thread-1-processing-n:127.0.0.1:43274_cn_jp%2Fy) [n:127.0.0.1:43274_cn_jp%2Fy c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=46, maxMergeAtOnceExplicit=21, maxMergedSegmentMB=72.4404296875, floorSegmentMB=1.3857421875, forceMergeDeletesPctAllowed=6.271570584226856, segmentsPerTier=25.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 3797941 WARN (coreLoadExecutor-4916-thread-1-processing-n:127.0.0.1:43274_cn_jp%2Fy) [n:127.0.0.1:43274_cn_jp%2Fy c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 3798068 INFO (coreLoadExecutor-4916-thread-1-processing-n:127.0.0.1:43274_cn_jp%2Fy) [n:127.0.0.1:43274_cn_jp%2Fy c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 3798068 INFO (coreLoadExecutor-4916-thread-1-processing-n:127.0.0.1:43274_cn_jp%2Fy) [n:127.0.0.1:43274_cn_jp%2Fy c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsTo
[...truncated too long message...]
java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
[junit4] > at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
[junit4] > at org.apache.solr.store.blockcache.BlockCache.<init>(BlockCache.java:75)
[junit4] > at org.apache.solr.core.HdfsDirectoryFactory.createBlockCache(HdfsDirectoryFactory.java:310)
[junit4] > ... 42 more
[junit4] 2> 9485704 WARN (SUITE-HdfsDirectoryFactoryTest-seed#[7024D709DA7A4582]-worker) [ ] o.a.h.h.s.d.DirectoryScanner DirectoryScanner: shutdown has been called
[junit4] 2> 9485814 INFO (SUITE-HdfsDirectoryFactoryTest-seed#[7024D709DA7A4582]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
[junit4] 2> 9485929 WARN (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.core.HdfsDirectoryFactoryTest_7024D709DA7A4582-001/tempDir-001/hdfsBaseDir/data/data3/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.core.HdfsDirectoryFactoryTest_7024D709DA7A4582-001/tempDir-001/hdfsBaseDir/data/data4/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:42363) [ ] o.a.h.h.s.d.IncrementalBlockReportManager IncrementalBlockReportManager interrupted
[junit4] 2> 9485929 WARN (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.core.HdfsDirectoryFactoryTest_7024D709DA7A4582-001/tempDir-001/hdfsBaseDir/data/data3/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.core.HdfsDirectoryFactoryTest_7024D709DA7A4582-001/tempDir-001/hdfsBaseDir/data/data4/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:42363) [ ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-17802106-127.0.0.1-1504429985210 (Datanode Uuid aee08116-a6ff-48d9-9e15-65ec8a82095d) service to lucene2-us-west.apache.org/127.0.0.1:42363
[junit4] 2> 9485931 WARN (SUITE-HdfsDirectoryFactoryTest-seed#[7024D709DA7A4582]-worker) [ ] o.a.h.h.s.d.DirectoryScanner DirectoryScanner: shutdown has been called
[junit4] 2> 9486028 INFO (SUITE-HdfsDirectoryFactoryTest-seed#[7024D709DA7A4582]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
[junit4] 2> 9486040 WARN (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.core.HdfsDirectoryFactoryTest_7024D709DA7A4582-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.core.HdfsDirectoryFactoryTest_7024D709DA7A4582-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:42363) [ ] o.a.h.h.s.d.IncrementalBlockReportManager IncrementalBlockReportManager interrupted
[junit4] 2> 9486041 WARN (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.core.HdfsDirectoryFactoryTest_7024D709DA7A4582-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.core.HdfsDirectoryFactoryTest_7024D709DA7A4582-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:42363) [ ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-17802106-127.0.0.1-1504429985210 (Datanode Uuid b08199a3-8179-4c5a-a6f3-77ac6a6f7484) service to lucene2-us-west.apache.org/127.0.0.1:42363
[junit4] 2> 9486096 INFO (SUITE-HdfsDirectoryFactoryTest-seed#[7024D709DA7A4582]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@lucene2-us-west.apache.org:0
[junit4] 2> 9486197 INFO (SUITE-HdfsDirectoryFactoryTest-seed#[7024D709DA7A4582]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.core.HdfsDirectoryFactoryTest_7024D709DA7A4582-001
[junit4] 2> Sep 03, 2017 9:13:14 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 34 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): {}, docValues:{}, maxPointsInLeafNode=469, maxMBSortInHeap=5.971039617496133, sim=RandomSimilarity(queryNorm=true,coord=yes): {}, locale=mt-MT, timezone=Africa/Dar_es_Salaam
[junit4] 2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=2,free=372199904,total=521666560
[junit4] 2> NOTE: All tests run in this JVM: [TestFieldCacheSort, CloneFieldUpdateProcessorFactoryTest, ClassificationUpdateProcessorFactoryTest, RestartWhileUpdatingTest, TestUseDocValuesAsStored2, TestScoreJoinQPNoScore, TestCopyFieldCollectionResource, TestSchemaManager, DirectSolrSpellCheckerTest, BooleanFieldTest, DistributedTermsComponentTest, DeleteReplicaTest, SharedFSAutoReplicaFailoverUtilsTest, TestCSVResponseWriter, CleanupOldIndexTest, SolrIndexSplitterTest, TestCustomDocTransformer, DistributedIntervalFacetingTest, TestCloudDeleteByQuery, TestLMDirichletSimilarityFactory, TestQueryUtils, TestManagedSchemaAPI, TestSQLHandlerNonCloud, CopyFieldTest, ShardRoutingCustomTest, MetricsHandlerTest, TestFiltering, LeaderInitiatedRecoveryOnShardRestartTest, CoreAdminRequestStatusTest, LeaderFailoverAfterPartitionTest, PeerSyncWithIndexFingerprintCachingTest, CSVRequestHandlerTest, TestRawResponseWriter, MultiTermTest, HdfsBasicDistributedZkTest, DistributedFacetPivotLargeTest, TestInPlaceUpdatesStandalone, SpellCheckComponentTest, OverseerTaskQueueTest, InfixSuggestersTest, HdfsWriteToMultipleCollectionsTest, TestDistribIDF, TestOrdValues, HighlighterTest, TestSystemCollAutoCreate, MoreLikeThisHandlerTest, TestReloadDeadlock, CursorMarkTest, IndexBasedSpellCheckerTest, TestElisionMultitermQuery, TestRestManager, SoftAutoCommitTest, CoreSorterTest, JSONWriterTest, TermsComponentTest, TestExclusionRuleCollectionAccess, TestCustomSort, DistributedQueueTest, HttpSolrCallGetCoreTest, TestInitParams, TestSweetSpotSimilarityFactory, TestDeleteCollectionOnDownNodes, JsonLoaderTest, TestDistributedGrouping, TestSolrDynamicMBean, CloudMLTQParserTest, IndexSchemaTest, CdcrBootstrapTest, OpenCloseCoreStressTest, PKIAuthenticationIntegrationTest, TestEmbeddedSolrServerConstructors, TestSystemIdResolver, TestConfigReload, UUIDUpdateProcessorFallbackTest, SearchHandlerTest, TestPerFieldSimilarityWithDefaultOverride, TestSegmentSorting, RemoteQueryErrorTest, TestTolerantUpdateProcessorCloud, TestMiniSolrCloudClusterSSL, SegmentsInfoRequestHandlerTest, TestIntervalFaceting, TestCoreContainer, CollectionStateFormat2Test, SimplePostToolTest, SolrJmxReporterCloudTest, BlockCacheTest, ConnectionReuseTest, TestConfigOverlay, ZkStateReaderTest, HdfsCollectionsAPIDistributedZkTest, TestNumericTerms64, SolrCoreMetricManagerTest, TestCloudPivotFacet, TestCursorMarkWithoutUniqueKey, TestSubQueryTransformerDistrib, SortByFunctionTest, SpellCheckCollatorTest, TestQueryTypes, TemplateUpdateProcessorTest, ForceLeaderTest, ZkControllerTest, DistanceFunctionTest, WordBreakSolrSpellCheckerTest, TestExceedMaxTermLength, TestFieldCollectionResource, TestSolrFieldCacheMBean, TestJsonFacetRefinement, TestSmileRequest, UpdateRequestProcessorFactoryTest, EchoParamsTest, TestLRUCache, DirectSolrConnectionTest, PreAnalyzedFieldManagedSchemaCloudTest, TestSimpleTrackingShardHandler, CdcrRequestHandlerTest, TestDistributedStatsComponentCardinality, ZkSolrClientTest, TestZkChroot, ZkCLITest, AutoCommitTest, TestRecovery, TestHashPartitioner, TermVectorComponentDistributedTest, TestStressVersions, TestRangeQuery, TestGroupingSearch, ConvertedLegacyTest, TestBadConfig, ShowFileRequestHandlerTest, TestCoreDiscovery, TestFoldingMultitermQuery, DocValuesTest, TestSurroundQueryParser, DocumentAnalysisRequestHandlerTest, TestOmitPositions, RequestHandlersTest, PathHierarchyTokenizerFactoryTest, TestIndexingPerformance, FastVectorHighlighterTest, LoggingHandlerTest, TestCollationField, MBeansHandlerTest, TestPartialUpdateDeduplication, TestComponentsName, SOLR749Test, TestSolrIndexConfig, BadComponentTest, TestMergePolicyConfig, TestBinaryField, NumericFieldsTest, OutputWriterTest, TestPhraseSuggestions, TestLMJelinekMercerSimilarityFactory, TestFastWriter, TestFastOutputStream, OpenExchangeRatesOrgProviderTest, PluginInfoTest, DateMathParserTest, PrimUtilsTest, TestSuggestSpellingConverter, SpellingQueryConverterTest, DOMUtilTest, RAMDirectoryFactoryTest, TestUtils, SliceStateTest, SystemInfoHandlerTest, CircularListTest, DistributedMLTComponentTest, TestTolerantSearch, TestPathTrie, TestEmbeddedSolrServerSchemaAPI, TestJettySolrRunner, AssignTest, CdcrReplicationDistributedZkTest, CloudExitableDirectoryReaderTest, CollectionReloadTest, CollectionTooManyReplicasTest, ConcurrentDeleteAndCreateCollectionTest, DistribCursorPagingTest, OutOfBoxZkACLAndCredentialsProvidersTest, OverseerModifyCollectionTest, OverseerStatusTest, RecoveryAfterSoftCommitTest, ReplaceNodeTest, ReplicationFactorTest, SaslZkACLProviderTest, SimpleCollectionCreateDeleteTest, TestCloudJSONFacetJoinDomain, TestConfigSetsAPI, TestRebalanceLeaders, TestRequestStatusCollectionAPI, TestSolrCloudWithDelegationTokens, TestStressCloudBlindAtomicUpdates, HdfsUnloadDistributedZkTest, TestClusterStateMutator, ZkStateWriterTest, RuleEngineTest, ConfigureRecoveryStrategyTest, DirectoryFactoryTest, HdfsDirectoryFactoryTest]
[junit4] Completed [554/713 (4!)] on J1 in 18.58s, 3 tests, 1 error <<< FAILURES!
[...truncated 43947 lines...]
[JENKINS] Lucene-Solr-NightlyTests-6.6 - Build # 28 - Failure
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.6/28/
9 tests failed.
FAILED: org.apache.lucene.spatial.geopoint.search.TestLegacyGeoPointQuery.testRandomBig
Error Message:
Java heap space
Stack Trace:
java.lang.OutOfMemoryError: Java heap space
at __randomizedtesting.SeedInfo.seed([721E11A0AE844B0E:F5496C2F3FDD378E]:0)
at org.apache.lucene.util.fst.BytesStore.writeByte(BytesStore.java:89)
at org.apache.lucene.util.fst.FST.pack(FST.java:1706)
at org.apache.lucene.util.fst.Builder.finish(Builder.java:500)
at org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsWriter.finish(MemoryPostingsFormat.java:263)
at org.apache.lucene.codecs.memory.MemoryPostingsFormat$MemoryFieldsConsumer.write(MemoryPostingsFormat.java:397)
at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:105)
at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.merge(PerFieldPostingsFormat.java:164)
at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:216)
at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:101)
at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4356)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3931)
at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2083)
at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1916)
at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1873)
at org.apache.lucene.geo.BaseGeoPointTestCase.verifyRandomPolygons(BaseGeoPointTestCase.java:1068)
at org.apache.lucene.geo.BaseGeoPointTestCase.verify(BaseGeoPointTestCase.java:756)
at org.apache.lucene.geo.BaseGeoPointTestCase.doTestRandom(BaseGeoPointTestCase.java:702)
at org.apache.lucene.geo.BaseGeoPointTestCase.testRandomBig(BaseGeoPointTestCase.java:633)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
FAILED: org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([8406B3D36E725FAA]:0)
FAILED: junit.framework.TestSuite.org.apache.lucene.spatial3d.TestGeo3DPoint
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([8406B3D36E725FAA]:0)
FAILED: org.apache.solr.cloud.CdcrReplicationHandlerTest.testReplicationWithBufferedUpdates
Error Message:
Timeout while trying to assert number of documents @ source_collection
Stack Trace:
java.lang.AssertionError: Timeout while trying to assert number of documents @ source_collection
at __randomizedtesting.SeedInfo.seed([D7F69022476FF3B8:4FFC03C02FC6F2F]:0)
at org.apache.solr.cloud.BaseCdcrDistributedZkTest.assertNumDocs(BaseCdcrDistributedZkTest.java:271)
at org.apache.solr.cloud.CdcrReplicationHandlerTest.testReplicationWithBufferedUpdates(CdcrReplicationHandlerTest.java:233)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.AssertionError: expected:<170> but was:<180>
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.BaseCdcrDistributedZkTest.assertNumDocs(BaseCdcrDistributedZkTest.java:262)
... 42 more
FAILED: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test
Error Message:
Captured an uncaught exception in thread: Thread[id=2329, name=Thread-645, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=2329, name=Thread-645, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
at __randomizedtesting.SeedInfo.seed([D7F69022476FF3B8:5FA2AFF8E9939E40]:0)
Caused by: org.apache.solr.common.SolrException: Could not load collection from ZK: collection2
at __randomizedtesting.SeedInfo.seed([D7F69022476FF3B8]:0)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1098)
at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:638)
at org.apache.solr.client.solrj.impl.CloudSolrClient.getDocCollection(CloudSolrClient.java:1489)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1099)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1073)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:177)
at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest$1IndexThread.run(FullSolrCloudDistribCmdsTest.java:641)
Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection2/state.json
at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1212)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:357)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:354)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:354)
at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1110)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1096)
... 7 more
FAILED: org.apache.solr.cloud.hdfs.HdfsSyncSliceTest.test
Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:46846/kcqg/collection1
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:46846/kcqg/collection1
at __randomizedtesting.SeedInfo.seed([D7F69022476FF3B8:5FA2AFF8E9939E40]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:623)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:484)
at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:463)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.commit(AbstractFullDistribZkTestBase.java:1529)
at org.apache.solr.cloud.SyncSliceTest.test(SyncSliceTest.java:183)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:515)
... 48 more
FAILED: org.apache.solr.security.BasicAuthIntegrationTest.testBasicAuth
Error Message:
Error from server at http://127.0.0.1:36889/solr/authCollection: No registered leader was found after waiting for 4000ms , collection: authCollection slice: shard2
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:36889/solr/authCollection: No registered leader was found after waiting for 4000ms , collection: authCollection slice: shard2
at __randomizedtesting.SeedInfo.seed([D7F69022476FF3B8:6B98E630E33C70C2]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:612)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:447)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:388)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1383)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1134)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1073)
at org.apache.solr.security.BasicAuthIntegrationTest.testBasicAuth(BasicAuthIntegrationTest.java:194)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.client.solrj.io.stream.StreamExpressionTest.testEvalStream
Error Message:
--> http://127.0.0.1:44893/solr/collection1:org.apache.solr.common.SolrException: Could not load collection from ZK: collection1_collection
Stack Trace:
java.io.IOException: --> http://127.0.0.1:44893/solr/collection1:org.apache.solr.common.SolrException: Could not load collection from ZK: collection1_collection
at __randomizedtesting.SeedInfo.seed([7280676B4D3E9B3F:3117A01D182B941E]:0)
at org.apache.solr.client.solrj.io.stream.SolrStream.read(SolrStream.java:233)
at org.apache.solr.client.solrj.io.stream.StreamExpressionTest.getTuples(StreamExpressionTest.java:6496)
at org.apache.solr.client.solrj.io.stream.StreamExpressionTest.testEvalStream(StreamExpressionTest.java:5133)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.hdfs.HdfsUnloadDistributedZkTest.test
Error Message:
Captured an uncaught exception in thread: Thread[id=116059, name=testExecutor-10509-thread-6, state=RUNNABLE, group=TGRP-HdfsUnloadDistributedZkTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=116059, name=testExecutor-10509-thread-6, state=RUNNABLE, group=TGRP-HdfsUnloadDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:45517: Error CREATEing SolrCore 'multiunload5': Unable to create core [multiunload5] Caused by: Direct buffer memory
at __randomizedtesting.SeedInfo.seed([D7F69022476FF3B8]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:612)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at org.apache.solr.cloud.BasicDistributedZkTest.lambda$createCollectionInOneInstance$0(BasicDistributedZkTest.java:589)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 10129 lines...]
[junit4] Suite: org.apache.lucene.spatial.geopoint.search.TestLegacyGeoPointQuery
[junit4] IGNOR/A 0.10s J1 | TestLegacyGeoPointQuery.testRandomDistance
[junit4] > Assumption #1: legacy encoding is too slow/hangs on this test
[junit4] IGNOR/A 0.01s J1 | TestLegacyGeoPointQuery.testSamePointManyTimes
[junit4] > Assumption #1: legacy encoding goes OOM on this test
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestLegacyGeoPointQuery -Dtests.method=testRandomBig -Dtests.seed=721E11A0AE844B0E -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/test-data/enwiki.random.lines.txt -Dtests.locale=el-GR -Dtests.timezone=Pacific/Bougainville -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 1313s J1 | TestLegacyGeoPointQuery.testRandomBig <<<
[junit4] > Throwable #1: java.lang.OutOfMemoryError: Java heap space
[junit4] > at __randomizedtesting.SeedInfo.seed([721E11A0AE844B0E:F5496C2F3FDD378E]:0)
[junit4] > at org.apache.lucene.util.fst.BytesStore.writeByte(BytesStore.java:89)
[junit4] > at org.apache.lucene.util.fst.FST.pack(FST.java:1706)
[junit4] > at org.apache.lucene.util.fst.Builder.finish(Builder.java:500)
[junit4] > at org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsWriter.finish(MemoryPostingsFormat.java:263)
[junit4] > at org.apache.lucene.codecs.memory.MemoryPostingsFormat$MemoryFieldsConsumer.write(MemoryPostingsFormat.java:397)
[junit4] > at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:105)
[junit4] > at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.merge(PerFieldPostingsFormat.java:164)
[junit4] > at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:216)
[junit4] > at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:101)
[junit4] > at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4356)
[junit4] > at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3931)
[junit4] > at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
[junit4] > at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2083)
[junit4] > at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1916)
[junit4] > at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1873)
[junit4] > at org.apache.lucene.geo.BaseGeoPointTestCase.verifyRandomPolygons(BaseGeoPointTestCase.java:1068)
[junit4] > at org.apache.lucene.geo.BaseGeoPointTestCase.verify(BaseGeoPointTestCase.java:756)
[junit4] > at org.apache.lucene.geo.BaseGeoPointTestCase.doTestRandom(BaseGeoPointTestCase.java:702)
[junit4] > at org.apache.lucene.geo.BaseGeoPointTestCase.testRandomBig(BaseGeoPointTestCase.java:633)
[junit4] IGNOR/A 0.01s J1 | TestLegacyGeoPointQuery.testRandomDistanceHuge
[junit4] > Assumption #1: legacy encoding is too slow/hangs on this test
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/lucene/build/spatial/test/J1/temp/lucene.spatial.geopoint.search.TestLegacyGeoPointQuery_721E11A0AE844B0E-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): {string=FST50, field=Lucene50(blocksize=128), id=PostingsFormat(name=LuceneVarGapFixedInterval), point=PostingsFormat(name=Memory doPackFST= true)}, docValues:{field=DocValuesFormat(name=Asserting), id=DocValuesFormat(name=Lucene54), point=DocValuesFormat(name=Lucene54)}, maxPointsInLeafNode=629, maxMBSortInHeap=7.065991696546367, sim=RandomSimilarity(queryNorm=false,coord=no): {}, locale=el-GR, timezone=Pacific/Bougainville
[junit4] 2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=320034232,total=455606272
[junit4] 2> NOTE: All tests run in this JVM: [TestLegacyGeoPointQuery]
[junit4] Completed [4/4 (1!)] on J1 in 1805.30s, 40 tests, 1 error, 3 skipped <<< FAILURES!
[...truncated 1 lines...]
[junit4] JVM J1: stdout was not empty, see: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/lucene/build/spatial/test/temp/junit4-J1-20170829_175625_5393488659577377724795.sysout
[junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
[junit4] java.lang.OutOfMemoryError: Java heap space
[junit4] Dumping heap to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/heapdumps/java_pid31180.hprof ...
[junit4] Heap dump file created [475029268 bytes in 3.274 secs]
[junit4] <<< JVM J1: EOF ----
[...truncated 284 lines...]
[junit4] Suite: org.apache.lucene.spatial3d.TestGeo3DPoint
[junit4] 2> Srp 30, 2017 12:26:42 DOP. com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
[junit4] 2> WARNING: Suite execution timed out: org.apache.lucene.spatial3d.TestGeo3DPoint
[junit4] 2> 1) Thread[id=12, name=TEST-TestGeo3DPoint.testRandomBig-seed#[8406B3D36E725FAA], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at org.apache.lucene.spatial3d.geom.Plane.findIntersections(Plane.java:829)
[junit4] 2> at org.apache.lucene.spatial3d.geom.Plane.findIntersections(Plane.java:650)
[junit4] 2> at org.apache.lucene.spatial3d.geom.StandardXYZSolid.<init>(StandardXYZSolid.java:152)
[junit4] 2> at org.apache.lucene.spatial3d.geom.XYZSolidFactory.makeXYZSolid(XYZSolidFactory.java:64)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.verify(TestGeo3DPoint.java:846)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.doTestRandom(TestGeo3DPoint.java:548)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig(TestGeo3DPoint.java:480)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 2) Thread[id=1, name=main, state=WAITING, group=main]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Thread.join(Thread.java:1252)
[junit4] 2> at java.lang.Thread.join(Thread.java:1326)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:608)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:457)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:244)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:355)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:13)
[junit4] 2> 3) Thread[id=11, name=SUITE-TestGeo3DPoint-seed#[8406B3D36E725FAA], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at java.lang.Thread.getStackTrace(Thread.java:1559)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:696)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:693)
[junit4] 2> at java.security.AccessController.doPrivileged(Native Method)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getStackTrace(ThreadLeakControl.java:693)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getThreadsWithTraces(ThreadLeakControl.java:709)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:689)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:65)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:415)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:678)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:140)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:598)
[junit4] 2> 4) Thread[id=9, name=JUnit4-serializer-daemon, state=TIMED_WAITING, group=main]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:50)
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestGeo3DPoint -Dtests.method=testRandomBig -Dtests.seed=8406B3D36E725FAA -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/test-data/enwiki.random.lines.txt -Dtests.locale=cs -Dtests.timezone=NET -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 7187s J1 | TestGeo3DPoint.testRandomBig <<<
[junit4] > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
[junit4] > at __randomizedtesting.SeedInfo.seed([8406B3D36E725FAA]:0)
[junit4] 2> Srp 30, 2017 12:26:42 DOP. com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> Srp 30, 2017 12:27:02 DOP. com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.lucene.spatial3d.TestGeo3DPoint:
[junit4] 2> 1) Thread[id=12, name=TEST-TestGeo3DPoint.testRandomBig-seed#[8406B3D36E725FAA], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoComplexPolygon.isWithin(GeoComplexPolygon.java:326)
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoBaseMembershipShape.isWithin(GeoBaseMembershipShape.java:36)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.verify(TestGeo3DPoint.java:848)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.doTestRandom(TestGeo3DPoint.java:548)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig(TestGeo3DPoint.java:480)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Srp 30, 2017 12:27:02 DOP. com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=12, name=TEST-TestGeo3DPoint.testRandomBig-seed#[8406B3D36E725FAA], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/lucene/build/spatial3d/test/J1/temp/lucene.spatial3d.TestGeo3DPoint_8406B3D36E725FAA-001
[junit4] 2> Aug 29, 2017 8:27:02 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: All leaked threads terminated.
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62), sim=RandomSimilarity(queryNorm=false,coord=yes): {}, locale=cs, timezone=NET
[junit4] 2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=149157264,total=491782144
[junit4] 2> NOTE: All tests run in this JVM: [TestGeo3DPoint]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestGeo3DPoint -Dtests.seed=8406B3D36E725FAA -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/test-data/enwiki.random.lines.txt -Dtests.locale=cs -Dtests.timezone=NET -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 0.00s J1 | TestGeo3DPoint (suite) <<<
[junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4] > at __randomizedtesting.SeedInfo.seed([8406B3D36E725FAA]:0)
[junit4] Completed [10/11 (1!)] on J1 in 7221.62s, 2 tests, 2 errors <<< FAILURES!
[...truncated 1811 lines...]
[junit4] Suite: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_D7F69022476FF3B8-001/init-core-data-001
[junit4] 2> 701876 WARN (SUITE-FullSolrCloudDistribCmdsTest-seed#[D7F69022476FF3B8]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=200 numCloses=200
[junit4] 2> 701876 INFO (SUITE-FullSolrCloudDistribCmdsTest-seed#[D7F69022476FF3B8]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields
[junit4] 2> 701877 INFO (SUITE-FullSolrCloudDistribCmdsTest-seed#[D7F69022476FF3B8]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
[junit4] 2> 701877 INFO (SUITE-FullSolrCloudDistribCmdsTest-seed#[D7F69022476FF3B8]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 2> 701899 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 701899 INFO (Thread-547) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 701899 INFO (Thread-547) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 701901 ERROR (Thread-547) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 701999 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.c.ZkTestServer start zk server on port:46194
[junit4] 2> 702100 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 702114 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 702115 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 702116 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 702116 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 702117 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 702132 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 702133 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 702146 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 702147 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 702148 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 702149 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e2fdaed890001, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 702557 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_D7F69022476FF3B8-001/control-001/cores/collection1
[junit4] 2> 702559 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 702634 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6ecfa2{/,null,AVAILABLE}
[junit4] 2> 702635 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3d732778{HTTP/1.1,[http/1.1]}{127.0.0.1:39208}
[junit4] 2> 702635 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.e.j.s.Server Started @714604ms
[junit4] 2> 702635 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_D7F69022476FF3B8-001/tempDir-001/control/data, hostContext=/, hostPort=39208, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_D7F69022476FF3B8-001/control-001/cores}
[junit4] 2> 702635 ERROR (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 702635 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.6.1
[junit4] 2> 702635 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 702635 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 702635 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-29T21:17:11.144Z
[junit4] 2> 702651 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 702651 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_D7F69022476FF3B8-001/control-001/solr.xml
[junit4] 2> 702674 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 702674 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46194/solr
[junit4] 2> 702772 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [n:127.0.0.1:39208_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 702784 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [n:127.0.0.1:39208_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:39208_
[junit4] 2> 702785 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [n:127.0.0.1:39208_ ] o.a.s.c.Overseer Overseer (id=98568859183284228-127.0.0.1:39208_-n_0000000000) starting
[junit4] 2> 702902 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [n:127.0.0.1:39208_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39208_
[junit4] 2> 702903 INFO (zkCallback-268-thread-1-processing-n:127.0.0.1:39208_) [n:127.0.0.1:39208_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 703159 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [n:127.0.0.1:39208_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_D7F69022476FF3B8-001/control-001/cores
[junit4] 2> 703159 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [n:127.0.0.1:39208_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 703203 INFO (coreLoadExecutor-579-thread-1-processing-n:127.0.0.1:39208_) [n:127.0.0.1:39208_ c:control_collection x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 703206 INFO (OverseerStateUpdate-98568859183284228-127.0.0.1:39208_-n_0000000000) [n:127.0.0.1:39208_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 704285 INFO (coreLoadExecutor-579-thread-1-processing-n:127.0.0.1:39208_) [n:127.0.0.1:39208_ c:control_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.1
[junit4] 2> 704334 INFO (coreLoadExecutor-579-thread-1-processing-n:127.0.0.1:39208_) [n:127.0.0.1:39208_ c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 704736 INFO (coreLoadExecutor-579-thread-1-processing-n:127.0.0.1:39208_) [n:127.0.0.1:39208_ c:control_collection x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 704786 INFO (coreLoadExecutor-579-thread-1-processing-n:127.0.0.1:39208_) [n:127.0.0.1:39208_ c:control_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection, trusted=true
[junit4] 2> 704786 INFO (coreLoadExecutor-579-thread-1-processing-n:127.0.0.1:39208_) [n:127.0.0.1:39208_ c:control_collection x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 704786 INFO (coreLoadExecutor-579-thread-1-processing-n:127.0.0.1:39208_) [n:127.0.0.1:39208_ c:control_collection x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_D7F69022476FF3B8-001/control-001/cores/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_D7F69022476FF3B8-001/control-001/cores/collection1/data/]
[junit4] 2> 704787 INFO (coreLoadExecutor-579-thread-1-processing-n:127.0.0.1:39208_) [n:127.0.0.1:39208_ c:control_collection x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ad6888e
[junit4] 2> 704788 INFO (coreLoadExecutor-579-thread-1-processing-n:127.0.0.1:39208_) [n:127.0.0.1:39208_ c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=28, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 704878 WARN (coreLoadExecutor-579-thread-1-processing-n:127.0.0.1:39208_) [n:127.0.0.1:39208_ c:control_collection x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 705009 INFO (coreLoadExecutor-579-thread-1-processing-n:127.0.0.1:39208_) [n:127.0.0.1:39208_ c:control_collection x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 705009 INFO (coreLoadExecutor-579-thread-1-processing-n:127.0.0.1:39208_) [n:127.0.0.1:39208_ c:control_collection x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 705010 INFO (coreLoadExecutor-579-thread-1-processing-n:127.0.0.1:39208_) [n:127.0.0.1:39208_ c:control_collection x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 705010 INFO (coreLoadExecutor-579-thread-1-processing-n:127.0.0.1:39208_) [n:127.0.0.1:39208_ c:control_collection x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 705011 INFO (coreLoadExecutor-579-thread-1-processing-n:127.0.0.1:39208_) [n:127.0.0.1:39208_ c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=39, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7804143450518743]
[junit4] 2> 705011 INFO (coreLoadExecutor-579-thread-1-processing-n:127.0.0.1:39208_) [n:127.0.0.1:39208_ c:control_collection x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@7b2d79d6[collection1] main]
[junit4] 2> 705037 INFO (coreLoadExecutor-579-thread-1-processing-n:127.0.0.1:39208_) [n:127.0.0.1:39208_ c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 705038 INFO (coreLoadExecutor-579-thread-1-processing-n:127.0.0.1:39208_) [n:127.0.0.1:39208_ c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 705038 INFO (coreLoadExecutor-579-thread-1-processing-n:127.0.0.1:39208_) [n:127.0.0.1:39208_ c:control_collection x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 705038 INFO (coreLoadExecutor-579-thread-1-processing-n:127.0.0.1:39208_) [n:127.0.0.1:39208_ c:control_collection x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577101750222979072
[junit4] 2> 705094 INFO (searcherExecutor-580-thread-1-processing-n:127.0.0.1:39208_ x:collection1 c:control_collection) [n:127.0.0.1:39208_ c:control_collection x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@7b2d79d6[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 705100 INFO (coreZkRegister-572-thread-1-processing-n:127.0.0.1:39208_ x:collection1 c:control_collection) [n:127.0.0.1:39208_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 705100 INFO (coreZkRegister-572-thread-1-processing-n:127.0.0.1:39208_ x:collection1 c:control_collection) [n:127.0.0.1:39208_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 705100 INFO (coreZkRegister-572-thread-1-processing-n:127.0.0.1:39208_ x:collection1 c:control_collection) [n:127.0.0.1:39208_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:39208/collection1/
[junit4] 2> 705100 INFO (coreZkRegister-572-thread-1-processing-n:127.0.0.1:39208_ x:collection1 c:control_collection) [n:127.0.0.1:39208_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 705100 INFO (coreZkRegister-572-thread-1-processing-n:127.0.0.1:39208_ x:collection1 c:control_collection) [n:127.0.0.1:39208_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:39208/collection1/ has no replicas
[junit4] 2> 705100 INFO (coreZkRegister-572-thread-1-processing-n:127.0.0.1:39208_ x:collection1 c:control_collection) [n:127.0.0.1:39208_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 705117 INFO (coreZkRegister-572-thread-1-processing-n:127.0.0.1:39208_ x:collection1 c:control_collection) [n:127.0.0.1:39208_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:39208/collection1/ shard1
[junit4] 2> 705224 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 705225 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:46194/solr ready
[junit4] 2> 705225 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 705225 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
[junit4] 2> 705292 INFO (coreZkRegister-572-thread-1-processing-n:127.0.0.1:39208_ x:collection1 c:control_collection) [n:127.0.0.1:39208_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 705540 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_D7F69022476FF3B8-001/shard-1-001/cores/collection1
[junit4] 2> 705557 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_D7F69022476FF3B8-001/shard-1-001
[junit4] 2> 705557 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 705558 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@57ece8c5{/,null,AVAILABLE}
[junit4] 2> 705559 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@73c2058b{HTTP/1.1,[http/1.1]}{127.0.0.1:37855}
[junit4] 2> 705559 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.e.j.s.Server Started @717528ms
[junit4] 2> 705559 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_D7F69022476FF3B8-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=37855, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_D7F69022476FF3B8-001/shard-1-001/cores}
[junit4] 2> 705559 ERROR (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 705560 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.6.1
[junit4] 2> 705560 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 705560 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 705560 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-29T21:17:14.069Z
[junit4] 2> 705601 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 705601 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_D7F69022476FF3B8-001/shard-1-001/solr.xml
[junit4] 2> 705630 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 705631 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46194/solr
[junit4] 2> 705653 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [n:127.0.0.1:37855_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 705667 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [n:127.0.0.1:37855_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 705669 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [n:127.0.0.1:37855_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37855_
[junit4] 2> 705681 INFO (zkCallback-278-thread-1-processing-n:127.0.0.1:37855_) [n:127.0.0.1:37855_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 705687 INFO (zkCallback-272-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 705687 INFO (zkCallback-268-thread-1-processing-n:127.0.0.1:39208_) [n:127.0.0.1:39208_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 706074 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [n:127.0.0.1:37855_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_D7F69022476FF3B8-001/shard-1-001/cores
[junit4] 2> 706074 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [n:127.0.0.1:37855_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 706090 INFO (coreLoadExecutor-590-thread-1-processing-n:127.0.0.1:37855_) [n:127.0.0.1:37855_ c:collection1 x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 706091 INFO (OverseerStateUpdate-98568859183284228-127.0.0.1:39208_-n_0000000000) [n:127.0.0.1:39208_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 706228 INFO (zkCallback-278-thread-1-processing-n:127.0.0.1:37855_) [n:127.0.0.1:37855_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 707132 INFO (coreLoadExecutor-590-thread-1-processing-n:127.0.0.1:37855_) [n:127.0.0.1:37855_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.1
[junit4] 2> 707190 INFO (coreLoadExecutor-590-thread-1-processing-n:127.0.0.1:37855_) [n:127.0.0.1:37855_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 707599 INFO (coreLoadExecutor-590-thread-1-processing-n:127.0.0.1:37855_) [n:127.0.0.1:37855_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 707691 INFO (coreLoadExecutor-590-thread-1-processing-n:127.0.0.1:37855_) [n:127.0.0.1:37855_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1, trusted=true
[junit4] 2> 707692 INFO (coreLoadExecutor-590-thread-1-processing-n:127.0.0.1:37855_) [n:127.0.0.1:37855_ c:collection1 x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 707692 INFO (coreLoadExecutor-590-thread-1-processing-n:127.0.0.1:37855_) [n:127.0.0.1:37855_ c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_D7F69022476FF3B8-001/shard-1-001/cores/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_D7F69022476FF3B8-001/shard-1-001/cores/collection1/data/]
[junit4] 2> 707692 INFO (coreLoadExecutor-590-thread-1-processing-n:127.0.0.1:37855_) [n:127.0.0.1:37855_ c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ad6888e
[junit4] 2> 707693 INFO (coreLoadExecutor-590-thread-1-processing-n:127.0.0.1:37855_) [n:127.0.0.1:37855_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=28, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 707748 WARN (coreLoadExecutor-590-thread-1-processing-n:127.0.0.1:37855_) [n:127.0.0.1:37855_ c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 707884 INFO (coreLoadExecutor-590-thread-1-processing-n:127.0.0.1:37855_) [n:127.0.0.1:37855_ c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 707884 INFO (coreLoadExecutor-590-thread-1-processing-n:127.0.0.1:37855_) [n:127.0.0.1:37855_ c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 707884 INFO (coreLoadExecutor-590-thread-1-processing-n:127.0.0.1:37855_) [n:127.0.0.1:37855_ c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 707884 INFO (coreLoadExecutor-590-thread-1-processing-n:127.0.0.1:37855_) [n:127.0.0.1:37855_ c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 707901 INFO (coreLoadExecutor-590-thread-1-processing-n:127.0.0.1:37855_) [n:127.0.0.1:37855_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=39, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7804143450518743]
[junit4] 2> 707902 INFO (coreLoadExecutor-590-thread-1-processing-n:127.0.0.1:37855_) [n:127.0.0.1:37855_ c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@367fd9a7[collection1] main]
[junit4] 2> 707906 INFO (coreLoadExecutor-590-thread-1-processing-n:127.0.0.1:37855_) [n:127.0.0.1:37855_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 707907 INFO (coreLoadExecutor-590-thread-1-processing-n:127.0.0.1:37855_) [n:127.0.0.1:37855_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 707907 INFO (coreLoadExecutor-590-thread-1-processing-n:127.0.0.1:37855_) [n:127.0.0.1:37855_ c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 707907 INFO (coreLoadExecutor-590-thread-1-processing-n:127.0.0.1:37855_) [n:127.0.0.1:37855_ c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577101753231343616
[junit4] 2> 707935 INFO (searcherExecutor-591-thread-1-processing-n:127.0.0.1:37855_ x:collection1 c:collection1) [n:127.0.0.1:37855_ c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@367fd9a7[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 707938 INFO (coreZkRegister-585-thread-1-processing-n:127.0.0.1:37855_ x:collection1 c:collection1) [n:127.0.0.1:37855_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 707938 INFO (coreZkRegister-585-thread-1-processing-n:127.0.0.1:37855_ x:collection1 c:collection1) [n:127.0.0.1:37855_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 707951 INFO (coreZkRegister-585-thread-1-processing-n:127.0.0.1:37855_ x:collection1 c:collection1) [n:127.0.0.1:37855_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:37855/collection1/
[junit4] 2> 707951 INFO (coreZkRegister-585-thread-1-processing-n:127.0.0.1:37855_ x:collection1 c:collection1) [n:127.0.0.1:37855_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 707951 INFO (coreZkRegister-585-thread-1-processing-n:127.0.0.1:37855_ x:collection1 c:collection1) [n:127.0.0.1:37855_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:37855/collection1/ has no replicas
[junit4] 2> 707951 INFO (coreZkRegister-585-thread-1-processing-n:127.0.0.1:37855_ x:collection1 c:collection1) [n:127.0.0.1:37855_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 707953 INFO (coreZkRegister-585-thread-1-processing-n:127.0.0.1:37855_ x:collection1 c:collection1) [n:127.0.0.1:37855_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:37855/collection1/ shard2
[junit4] 2> 708055 INFO (zkCallback-278-thread-1-processing-n:127.0.0.1:37855_) [n:127.0.0.1:37855_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 708103 INFO (coreZkRegister-585-thread-1-processing-n:127.0.0.1:37855_ x:collection1 c:collection1) [n:127.0.0.1:37855_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 708246 INFO (zkCallback-278-thread-1-processing-n:127.0.0.1:37855_) [n:127.0.0.1:37855_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 708506 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_D7F69022476FF3B8-001/shard-2-001/cores/collection1
[junit4] 2> 708507 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_D7F69022476FF3B8-001/shard-2-001
[junit4] 2> 708520 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 708521 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@77cbbe25{/,null,AVAILABLE}
[junit4] 2> 708522 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@40bea512{HTTP/1.1,[http/1.1]}{127.0.0.1:39214}
[junit4] 2> 708522 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.e.j.s.Server Started @720491ms
[junit4] 2> 708522 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_D7F69022476FF3B8-001/tempDir-001/jetty2, solrconfig=solrconfig.xml, hostContext=/, hostPort=39214, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_D7F69022476FF3B8-001/shard-2-001/cores}
[junit4] 2> 708522 ERROR (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 708523 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.6.1
[junit4] 2> 708523 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 708523 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 708523 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-29T21:17:17.032Z
[junit4] 2> 708567 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 708567 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_D7F69022476FF3B8-001/shard-2-001/solr.xml
[junit4] 2> 708598 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 708599 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46194/solr
[junit4] 2> 708628 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [n:127.0.0.1:39214_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 708639 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [n:127.0.0.1:39214_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 708641 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[D7F69022476FF3B8]) [n:127.0.0.1:39214_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39214_
[junit4] 2> 708679 INFO (zkCallback-278-thread-1-processing-n:127.0.0.1:37855_) [n:127.0.0.1:37855_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 708687 INFO (zkCallback-268-thread-2-processing-n:127.0.0.1:39208_) [n:127.0.0.1:39208_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 708688 INFO (zkCallback-284-thread-1-processing-n:127.0.0.1:39214_) [n:127.0.0.1:39214_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[...truncated too long message...]
porters for: solr.core.collection1_collection.shard2.replica1
[junit4] 2> 416340 INFO (zkCallback-57-thread-3-processing-n:127.0.0.1:44893_solr) [n:127.0.0.1:44893_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 416376 INFO (zkCallback-1410-thread-1-processing-n:127.0.0.1:33922_solr x:collection1_collection_shard2_replica1 s:shard2 c:collection1_collection r:core_node1) [n:127.0.0.1:33922_solr c:collection1_collection s:shard2 r:core_node1 x:collection1_collection_shard2_replica1] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 416396 INFO (coreCloseExecutor-472-thread-1) [n:127.0.0.1:44893_solr c:collection1_collection s:shard1 r:core_node2 x:collection1_collection_shard1_replica1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.collection1_collection.shard1.replica1
[junit4] 2> 416460 INFO (zkCallback-57-thread-3-processing-n:127.0.0.1:44893_solr) [n:127.0.0.1:44893_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 416461 INFO (zkCallback-55-thread-4-processing-n:127.0.0.1:43164_solr) [n:127.0.0.1:43164_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 416480 INFO (jetty-closer-37-thread-3) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@d05d262{/solr,null,UNAVAILABLE}
[junit4] 2> 416481 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e307fe6f80009, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 416494 INFO (zkCallback-57-thread-2-processing-n:127.0.0.1:44893_solr) [n:127.0.0.1:44893_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:44893_solr
[junit4] 2> 416494 INFO (zkCallback-57-thread-3-processing-n:127.0.0.1:44893_solr) [n:127.0.0.1:44893_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 416543 WARN (zkCallback-57-thread-3-processing-n:127.0.0.1:44893_solr) [n:127.0.0.1:44893_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 417972 WARN (zkCallback-58-thread-4-processing-n:127.0.0.1:33922_solr) [n:127.0.0.1:33922_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 417972 INFO (jetty-closer-37-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1bcf0a77{/solr,null,UNAVAILABLE}
[junit4] 2> 418005 INFO (jetty-closer-37-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@78b1b70d{/solr,null,UNAVAILABLE}
[junit4] 2> 418058 ERROR (SUITE-StreamExpressionTest-seed#[7280676B4D3E9B3F]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 418059 INFO (SUITE-StreamExpressionTest-seed#[7280676B4D3E9B3F]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:33573 33573
[junit4] 2> 418395 INFO (Thread-34) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:33573 33573
[junit4] 2> 418397 WARN (Thread-34) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 66 /solr/aliases.json
[junit4] 2> 66 /solr/clusterprops.json
[junit4] 2> 10 /solr/configs/conf
[junit4] 2> 4 /solr/security.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 66 /solr/clusterstate.json
[junit4] 2> 4 /solr/collections/destination/state.json
[junit4] 2> 4 /solr/collections/mainCorpus/state.json
[junit4] 2> 3 /solr/collections/parallelDestinationCollection/state.json
[junit4] 2> 3 /solr/collections/destinationCollection/state.json
[junit4] 2> 2 /solr/collections/collection1_collection/state.json
[junit4] 2> 2 /solr/collections/workQueue/state.json
[junit4] 2> 2 /solr/collections/parallelDestinationCollection1/state.json
[junit4] 2> 2 /solr/overseer_elect/election/98569567742722059-127.0.0.1:44893_solr-n_0000000001
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 66 /solr/live_nodes
[junit4] 2> 66 /solr/collections
[junit4] 2>
[junit4] 2> 418398 INFO (SUITE-StreamExpressionTest-seed#[7280676B4D3E9B3F]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.stream.StreamExpressionTest_7280676B4D3E9B3F-001
[junit4] 2> Aug 30, 2017 12:22:11 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): {expr_s=PostingsFormat(name=Memory doPackFST= true), join2_s=PostingsFormat(name=Memory doPackFST= false), multiDefault=PostingsFormat(name=Memory doPackFST= false), subject=PostingsFormat(name=Memory doPackFST= false), a1_s=PostingsFormat(name=Memory doPackFST= false), price_f=PostingsFormat(name=Memory doPackFST= false), a_f=PostingsFormat(name=Memory doPackFST= false), intDefault=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), a2_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), join1_i=PostingsFormat(name=Memory doPackFST= true), a_i=PostingsFormat(name=Memory doPackFST= true), join3_i=PostingsFormat(name=LuceneVarGapDocFreqInterval), test_i=PostingsFormat(name=Memory doPackFST= false), field_i=PostingsFormat(name=Memory doPackFST= false), level1_s=PostingsFormat(name=Memory doPackFST= true), body_t=PostingsFormat(name=LuceneVarGapDocFreqInterval), s_multi=PostingsFormat(name=LuceneVarGapDocFreqInterval), col_s=PostingsFormat(name=Memory doPackFST= false), a_ss=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), level2_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), a_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), id=PostingsFormat(name=Memory doPackFST= true), text=PostingsFormat(name=LuceneVarGapDocFreqInterval), timestamp=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), i_multi=PostingsFormat(name=Memory doPackFST= true), b_ls=PostingsFormat(name=Memory doPackFST= false), test_dt=PostingsFormat(name=Memory doPackFST= true), test_t=PostingsFormat(name=LuceneVarGapDocFreqInterval), ident_s=PostingsFormat(name=Memory doPackFST= false), order_i=PostingsFormat(name=Memory doPackFST= false), _version_=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), side_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), miles_i=PostingsFormat(name=Memory doPackFST= false)}, docValues:{expr_s=DocValuesFormat(name=Lucene54), join2_s=DocValuesFormat(name=Lucene54), multiDefault=DocValuesFormat(name=Lucene54), a1_s=DocValuesFormat(name=Lucene54), price_f=DocValuesFormat(name=Lucene54), a_f=DocValuesFormat(name=Lucene54), intDefault=DocValuesFormat(name=Memory), a2_s=DocValuesFormat(name=Direct), join1_i=DocValuesFormat(name=Lucene54), a_i=DocValuesFormat(name=Lucene54), join3_i=DocValuesFormat(name=Direct), test_i=DocValuesFormat(name=Lucene54), field_i=DocValuesFormat(name=Lucene54), level1_s=DocValuesFormat(name=Lucene54), s_multi=DocValuesFormat(name=Direct), col_s=DocValuesFormat(name=Lucene54), a_ss=DocValuesFormat(name=Memory), level2_s=DocValuesFormat(name=Memory), a_s=DocValuesFormat(name=Direct), id=DocValuesFormat(name=Lucene54), timestamp=DocValuesFormat(name=Memory), i_multi=DocValuesFormat(name=Lucene54), b_ls=DocValuesFormat(name=Lucene54), test_dt=DocValuesFormat(name=Lucene54), ident_s=DocValuesFormat(name=Lucene54), order_i=DocValuesFormat(name=Lucene54), _version_=DocValuesFormat(name=Memory), side_s=DocValuesFormat(name=Direct), miles_i=DocValuesFormat(name=Lucene54)}, maxPointsInLeafNode=83, maxMBSortInHeap=7.875960485661203, sim=RandomSimilarity(queryNorm=true,coord=crazy): {}, locale=sv, timezone=Brazil/East
[junit4] 2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=171102128,total=477102080
[junit4] 2> NOTE: All tests run in this JVM: [SolrParamTest, ConcurrentUpdateSolrClientTest, HttpClientUtilTest, LessThanEvaluatorTest, TestBatchUpdate, CommonParamsTest, TestCoreAdmin, ArcTangentEvaluatorTest, TestRetryUtil, AndEvaluatorTest, TestSuggesterResponse, StreamExpressionTest]
[junit4] Completed [84/130 (1!)] on J2 in 291.32s, 70 tests, 1 error, 9 skipped <<< FAILURES!
[...truncated 6807 lines...]
BUILD FAILED
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/build.xml:783: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/build.xml:727: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid31180.hprof
Total time: 777 minutes 36 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-NightlyTests-6.6 - Build # 27 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.6/27/
5 tests failed.
FAILED: org.apache.solr.cloud.RollingRestartTest.test
Error Message:
KeeperErrorCode = Session expired for /overseer_elect/leader
Stack Trace:
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
at __randomizedtesting.SeedInfo.seed([7B4EC0BE997D7FBF:F31AFF6437811247]:0)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1212)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:357)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:354)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:354)
at org.apache.solr.cloud.OverseerTaskProcessor.getLeaderId(OverseerTaskProcessor.java:374)
at org.apache.solr.cloud.OverseerTaskProcessor.getLeaderNode(OverseerTaskProcessor.java:365)
at org.apache.solr.cloud.RollingRestartTest.waitUntilOverseerDesignateIsLeader(RollingRestartTest.java:137)
at org.apache.solr.cloud.RollingRestartTest.restartWithRolesTest(RollingRestartTest.java:104)
at org.apache.solr.cloud.RollingRestartTest.test(RollingRestartTest.java:52)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.testCollectionsAPI
Error Message:
Something is broken in the assert for no shards using the same indexDir - probably something was changed in the attributes published in the MBean of SolrCore : {}
Stack Trace:
java.lang.AssertionError: Something is broken in the assert for no shards using the same indexDir - probably something was changed in the attributes published in the MBean of SolrCore : {}
at __randomizedtesting.SeedInfo.seed([7B4EC0BE997D7FBF:333BB40A9F4E502A]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.checkNoTwoShardsUseTheSameIndexDir(CollectionsAPIDistributedZkTest.java:646)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:524)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.hdfs.HdfsUnloadDistributedZkTest.test
Error Message:
Captured an uncaught exception in thread: Thread[id=5450, name=testExecutor-1543-thread-3, state=RUNNABLE, group=TGRP-HdfsUnloadDistributedZkTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=5450, name=testExecutor-1543-thread-3, state=RUNNABLE, group=TGRP-HdfsUnloadDistributedZkTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:38844
at __randomizedtesting.SeedInfo.seed([7B4EC0BE997D7FBF]:0)
at org.apache.solr.cloud.UnloadDistributedZkTest.lambda$testUnloadLotsOfCores$0(UnloadDistributedZkTest.java:412)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:38844
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:623)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at org.apache.solr.cloud.UnloadDistributedZkTest.lambda$testUnloadLotsOfCores$0(UnloadDistributedZkTest.java:410)
... 4 more
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:515)
... 8 more
FAILED: org.apache.solr.update.AutoCommitTest.testMaxTime
Error Message:
Exception during query
Stack Trace:
java.lang.RuntimeException: Exception during query
at __randomizedtesting.SeedInfo.seed([7B4EC0BE997D7FBF:E1BABD5C07E7E383]:0)
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:895)
at org.apache.solr.update.AutoCommitTest.testMaxTime(AutoCommitTest.java:270)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
</response>
request was:q=id:529&qt=standard&start=0&rows=20&version=2.2
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:888)
... 40 more
FAILED: org.apache.solr.cloud.hdfs.StressHdfsTest.test
Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:36852/hs
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:36852/hs
at __randomizedtesting.SeedInfo.seed([7B4EC0BE997D7FBF:F31AFF6437811247]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:623)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:447)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:388)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1383)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1134)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1073)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at org.apache.solr.cloud.hdfs.StressHdfsTest.createAndDeleteCollection(StressHdfsTest.java:220)
at org.apache.solr.cloud.hdfs.StressHdfsTest.test(StressHdfsTest.java:103)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:515)
... 51 more
Build Log:
[...truncated 12278 lines...]
[junit4] Suite: org.apache.solr.cloud.hdfs.HdfsUnloadDistributedZkTest
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_7B4EC0BE997D7FBF-001/init-core-data-001
[junit4] 2> 1269296 WARN (SUITE-HdfsUnloadDistributedZkTest-seed#[7B4EC0BE997D7FBF]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=12 numCloses=12
[junit4] 2> 1269296 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[7B4EC0BE997D7FBF]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields
[junit4] 2> 1269309 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[7B4EC0BE997D7FBF]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
[junit4] 2> 1269310 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[7B4EC0BE997D7FBF]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 2> 1272232 WARN (SUITE-HdfsUnloadDistributedZkTest-seed#[7B4EC0BE997D7FBF]-worker) [ ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 1277134 WARN (SUITE-HdfsUnloadDistributedZkTest-seed#[7B4EC0BE997D7FBF]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 1277848 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[7B4EC0BE997D7FBF]-worker) [ ] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
[junit4] 2> 1277943 WARN (SUITE-HdfsUnloadDistributedZkTest-seed#[7B4EC0BE997D7FBF]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 1279411 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[7B4EC0BE997D7FBF]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 1279754 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[7B4EC0BE997D7FBF]-worker) [ ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/hdfs to ./temp/Jetty_lucene2.us.west_apache_org_37765_hdfs____dgfa9o/webapp
[junit4] 2> 1282652 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[7B4EC0BE997D7FBF]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@lucene2-us-west.apache.org:37765
[junit4] 2> 1286443 WARN (SUITE-HdfsUnloadDistributedZkTest-seed#[7B4EC0BE997D7FBF]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 1286447 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[7B4EC0BE997D7FBF]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 1286552 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[7B4EC0BE997D7FBF]-worker) [ ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_39226_datanode____.a679s9/webapp
[junit4] 2> 1288035 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[7B4EC0BE997D7FBF]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:39226
[junit4] 2> 1290888 WARN (SUITE-HdfsUnloadDistributedZkTest-seed#[7B4EC0BE997D7FBF]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 1290889 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[7B4EC0BE997D7FBF]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 1291025 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[7B4EC0BE997D7FBF]-worker) [ ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_43840_datanode____breblo/webapp
[junit4] 2> 1292681 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[7B4EC0BE997D7FBF]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43840
[junit4] 2> 1296380 ERROR (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_7B4EC0BE997D7FBF-001/tempDir-001/hdfsBaseDir/data/data3/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_7B4EC0BE997D7FBF-001/tempDir-001/hdfsBaseDir/data/data4/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:44350) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 1296394 ERROR (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_7B4EC0BE997D7FBF-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_7B4EC0BE997D7FBF-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:44350) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 1297218 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0xdb7f9a3d3604b: from storage DS-5c673ec8-4a98-40f2-a0ec-f0690acbd68a node DatanodeRegistration(127.0.0.1:35179, datanodeUuid=6bde4e5b-85f5-4024-bce3-ccdcccae1fba, infoPort=34534, infoSecurePort=0, ipcPort=33301, storageInfo=lv=-56;cid=testClusterID;nsid=1058005071;c=0), blocks: 0, hasStaleStorage: true, processing time: 18 msecs
[junit4] 2> 1297219 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0xdb7f9a284ed93: from storage DS-918d05a7-c681-4c5b-a0ef-137bd545f87a node DatanodeRegistration(127.0.0.1:34736, datanodeUuid=1f4cd606-9273-4be0-b981-a739a6650f23, infoPort=42562, infoSecurePort=0, ipcPort=34525, storageInfo=lv=-56;cid=testClusterID;nsid=1058005071;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 1297219 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0xdb7f9a284ed93: from storage DS-bc859f4c-008f-4ba7-811b-b08bd183ce61 node DatanodeRegistration(127.0.0.1:34736, datanodeUuid=1f4cd606-9273-4be0-b981-a739a6650f23, infoPort=42562, infoSecurePort=0, ipcPort=34525, storageInfo=lv=-56;cid=testClusterID;nsid=1058005071;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 1297221 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0xdb7f9a3d3604b: from storage DS-c2c393ee-d247-413f-8738-7b91544930ab node DatanodeRegistration(127.0.0.1:35179, datanodeUuid=6bde4e5b-85f5-4024-bce3-ccdcccae1fba, infoPort=34534, infoSecurePort=0, ipcPort=33301, storageInfo=lv=-56;cid=testClusterID;nsid=1058005071;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 1299059 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1299059 INFO (Thread-1139) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1299059 INFO (Thread-1139) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1299067 ERROR (Thread-1139) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1299159 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.c.ZkTestServer start zk server on port:35961
[junit4] 2> 1299341 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 1299342 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 1299343 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1299344 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 1299357 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 1299358 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 1299359 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 1299360 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 1299360 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1299370 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 1299371 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 1299376 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e1e9cdcaa0001, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 1299743 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_7B4EC0BE997D7FBF-001/control-001/cores/collection1
[junit4] 2> 1299801 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1299825 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3f276eed{/,null,AVAILABLE}
[junit4] 2> 1299826 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@4d6f31d4{HTTP/1.1,[http/1.1]}{127.0.0.1:41989}
[junit4] 2> 1299826 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.e.j.s.Server Started @1312648ms
[junit4] 2> 1299826 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://lucene2-us-west.apache.org:44350/hdfs__lucene2-us-west.apache.org_44350__home_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-6.6_checkout_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.HdfsUnloadDistributedZkTest_7B4EC0BE997D7FBF-001_tempDir-002_control_data, hostContext=/, hostPort=41989, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_7B4EC0BE997D7FBF-001/control-001/cores}
[junit4] 2> 1299826 ERROR (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1299826 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.6.1
[junit4] 2> 1299826 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1299826 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1299826 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-26T12:55:50.952Z
[junit4] 2> 1299894 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1299894 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_7B4EC0BE997D7FBF-001/control-001/solr.xml
[junit4] 2> 1299955 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 1299955 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35961/solr
[junit4] 2> 1300207 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [n:127.0.0.1:41989_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1300208 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [n:127.0.0.1:41989_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:41989_
[junit4] 2> 1300208 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [n:127.0.0.1:41989_ ] o.a.s.c.Overseer Overseer (id=98549900914589700-127.0.0.1:41989_-n_0000000000) starting
[junit4] 2> 1300317 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [n:127.0.0.1:41989_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41989_
[junit4] 2> 1300353 INFO (zkCallback-765-thread-1-processing-n:127.0.0.1:41989_) [n:127.0.0.1:41989_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1300645 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [n:127.0.0.1:41989_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_7B4EC0BE997D7FBF-001/control-001/cores
[junit4] 2> 1300645 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [n:127.0.0.1:41989_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1300646 INFO (coreLoadExecutor-1404-thread-1-processing-n:127.0.0.1:41989_) [n:127.0.0.1:41989_ c:control_collection x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 1300678 INFO (OverseerStateUpdate-98549900914589700-127.0.0.1:41989_-n_0000000000) [n:127.0.0.1:41989_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 1301806 WARN (coreLoadExecutor-1404-thread-1-processing-n:127.0.0.1:41989_) [n:127.0.0.1:41989_ c:control_collection x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 1301921 INFO (coreLoadExecutor-1404-thread-1-processing-n:127.0.0.1:41989_) [n:127.0.0.1:41989_ c:control_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.1
[junit4] 2> 1302002 INFO (coreLoadExecutor-1404-thread-1-processing-n:127.0.0.1:41989_) [n:127.0.0.1:41989_ c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1303467 INFO (coreLoadExecutor-1404-thread-1-processing-n:127.0.0.1:41989_) [n:127.0.0.1:41989_ c:control_collection x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 1303523 INFO (coreLoadExecutor-1404-thread-1-processing-n:127.0.0.1:41989_) [n:127.0.0.1:41989_ c:control_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection, trusted=true
[junit4] 2> 1303690 INFO (coreLoadExecutor-1404-thread-1-processing-n:127.0.0.1:41989_) [n:127.0.0.1:41989_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://lucene2-us-west.apache.org:44350/solr_hdfs_home
[junit4] 2> 1303691 INFO (coreLoadExecutor-1404-thread-1-processing-n:127.0.0.1:41989_) [n:127.0.0.1:41989_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 1303691 INFO (coreLoadExecutor-1404-thread-1-processing-n:127.0.0.1:41989_) [n:127.0.0.1:41989_ c:control_collection x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1303691 INFO (coreLoadExecutor-1404-thread-1-processing-n:127.0.0.1:41989_) [n:127.0.0.1:41989_ c:control_collection x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_7B4EC0BE997D7FBF-001/control-001/cores/collection1], dataDir=[hdfs://lucene2-us-west.apache.org:44350/solr_hdfs_home/control_collection/core_node1/data/]
[junit4] 2> 1303691 INFO (coreLoadExecutor-1404-thread-1-processing-n:127.0.0.1:41989_) [n:127.0.0.1:41989_ c:control_collection x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@af328a6
[junit4] 2> 1303692 INFO (coreLoadExecutor-1404-thread-1-processing-n:127.0.0.1:41989_) [n:127.0.0.1:41989_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:44350/solr_hdfs_home/control_collection/core_node1/data/snapshot_metadata
[junit4] 2> 1303858 INFO (coreLoadExecutor-1404-thread-1-processing-n:127.0.0.1:41989_) [n:127.0.0.1:41989_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1303858 INFO (coreLoadExecutor-1404-thread-1-processing-n:127.0.0.1:41989_) [n:127.0.0.1:41989_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 1303858 INFO (coreLoadExecutor-1404-thread-1-processing-n:127.0.0.1:41989_) [n:127.0.0.1:41989_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new global HDFS BlockCache
[junit4] 2> 1306927 INFO (coreLoadExecutor-1404-thread-1-processing-n:127.0.0.1:41989_) [n:127.0.0.1:41989_ c:control_collection x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1306943 INFO (coreLoadExecutor-1404-thread-1-processing-n:127.0.0.1:41989_) [n:127.0.0.1:41989_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:44350/solr_hdfs_home/control_collection/core_node1/data
[junit4] 2> 1307288 INFO (coreLoadExecutor-1404-thread-1-processing-n:127.0.0.1:41989_) [n:127.0.0.1:41989_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:44350/solr_hdfs_home/control_collection/core_node1/data/index
[junit4] 2> 1307340 INFO (coreLoadExecutor-1404-thread-1-processing-n:127.0.0.1:41989_) [n:127.0.0.1:41989_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1307340 INFO (coreLoadExecutor-1404-thread-1-processing-n:127.0.0.1:41989_) [n:127.0.0.1:41989_ c:control_collection x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 1307404 INFO (coreLoadExecutor-1404-thread-1-processing-n:127.0.0.1:41989_) [n:127.0.0.1:41989_ c:control_collection x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1307405 INFO (coreLoadExecutor-1404-thread-1-processing-n:127.0.0.1:41989_) [n:127.0.0.1:41989_ c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@3cfbc5cf
[junit4] 2> 1309305 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:34736 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-c2c393ee-d247-413f-8738-7b91544930ab:NORMAL:127.0.0.1:35179|RBW], ReplicaUC[[DISK]DS-bc859f4c-008f-4ba7-811b-b08bd183ce61:NORMAL:127.0.0.1:34736|RBW]]} size 0
[junit4] 2> 1309423 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:35179 is added to blk_1073741825_1001 size 71
[junit4] 2> 1309634 WARN (coreLoadExecutor-1404-thread-1-processing-n:127.0.0.1:41989_) [n:127.0.0.1:41989_ c:control_collection x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1310014 INFO (coreLoadExecutor-1404-thread-1-processing-n:127.0.0.1:41989_) [n:127.0.0.1:41989_ c:control_collection x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 1310014 INFO (coreLoadExecutor-1404-thread-1-processing-n:127.0.0.1:41989_) [n:127.0.0.1:41989_ c:control_collection x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1310015 INFO (coreLoadExecutor-1404-thread-1-processing-n:127.0.0.1:41989_) [n:127.0.0.1:41989_ c:control_collection x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 1310146 INFO (coreLoadExecutor-1404-thread-1-processing-n:127.0.0.1:41989_) [n:127.0.0.1:41989_ c:control_collection x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1310146 INFO (coreLoadExecutor-1404-thread-1-processing-n:127.0.0.1:41989_) [n:127.0.0.1:41989_ c:control_collection x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1310148 INFO (coreLoadExecutor-1404-thread-1-processing-n:127.0.0.1:41989_) [n:127.0.0.1:41989_ c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=9, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6603457310830974]
[junit4] 2> 1310671 INFO (coreLoadExecutor-1404-thread-1-processing-n:127.0.0.1:41989_) [n:127.0.0.1:41989_ c:control_collection x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@c89e940[collection1] main]
[junit4] 2> 1310749 INFO (coreLoadExecutor-1404-thread-1-processing-n:127.0.0.1:41989_) [n:127.0.0.1:41989_ c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1310765 INFO (coreLoadExecutor-1404-thread-1-processing-n:127.0.0.1:41989_) [n:127.0.0.1:41989_ c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1310767 INFO (coreLoadExecutor-1404-thread-1-processing-n:127.0.0.1:41989_) [n:127.0.0.1:41989_ c:control_collection x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1310785 INFO (searcherExecutor-1405-thread-1-processing-n:127.0.0.1:41989_ x:collection1 c:control_collection) [n:127.0.0.1:41989_ c:control_collection x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@c89e940[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1310832 INFO (coreLoadExecutor-1404-thread-1-processing-n:127.0.0.1:41989_) [n:127.0.0.1:41989_ c:control_collection x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1576798426977271808
[junit4] 2> 1310918 INFO (coreZkRegister-1397-thread-1-processing-n:127.0.0.1:41989_ x:collection1 c:control_collection) [n:127.0.0.1:41989_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1310918 INFO (coreZkRegister-1397-thread-1-processing-n:127.0.0.1:41989_ x:collection1 c:control_collection) [n:127.0.0.1:41989_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1310918 INFO (coreZkRegister-1397-thread-1-processing-n:127.0.0.1:41989_ x:collection1 c:control_collection) [n:127.0.0.1:41989_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:41989/collection1/
[junit4] 2> 1310918 INFO (coreZkRegister-1397-thread-1-processing-n:127.0.0.1:41989_ x:collection1 c:control_collection) [n:127.0.0.1:41989_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1310918 INFO (coreZkRegister-1397-thread-1-processing-n:127.0.0.1:41989_ x:collection1 c:control_collection) [n:127.0.0.1:41989_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:41989/collection1/ has no replicas
[junit4] 2> 1310918 INFO (coreZkRegister-1397-thread-1-processing-n:127.0.0.1:41989_ x:collection1 c:control_collection) [n:127.0.0.1:41989_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 1310937 INFO (coreZkRegister-1397-thread-1-processing-n:127.0.0.1:41989_ x:collection1 c:control_collection) [n:127.0.0.1:41989_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:41989/collection1/ shard1
[junit4] 2> 1310997 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1310997 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:35961/solr ready
[junit4] 2> 1310997 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 1311179 INFO (coreZkRegister-1397-thread-1-processing-n:127.0.0.1:41989_ x:collection1 c:control_collection) [n:127.0.0.1:41989_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1311636 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_7B4EC0BE997D7FBF-001/shard-1-001/cores/collection1
[junit4] 2> 1311636 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_7B4EC0BE997D7FBF-001/shard-1-001
[junit4] 2> 1311716 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1311765 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@429c489b{/,null,AVAILABLE}
[junit4] 2> 1311766 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@216ac074{HTTP/1.1,[http/1.1]}{127.0.0.1:38844}
[junit4] 2> 1311766 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.e.j.s.Server Started @1324588ms
[junit4] 2> 1311766 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://lucene2-us-west.apache.org:44350/hdfs__lucene2-us-west.apache.org_44350__home_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-6.6_checkout_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.HdfsUnloadDistributedZkTest_7B4EC0BE997D7FBF-001_tempDir-002_jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=38844, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_7B4EC0BE997D7FBF-001/shard-1-001/cores}
[junit4] 2> 1311766 ERROR (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1311801 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.6.1
[junit4] 2> 1311801 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1311801 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1311801 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-26T12:56:02.927Z
[junit4] 2> 1311867 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1311867 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_7B4EC0BE997D7FBF-001/shard-1-001/solr.xml
[junit4] 2> 1311971 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 1311992 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35961/solr
[junit4] 2> 1312214 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [n:127.0.0.1:38844_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1312260 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [n:127.0.0.1:38844_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1312261 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [n:127.0.0.1:38844_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38844_
[junit4] 2> 1312263 INFO (zkCallback-769-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1312280 INFO (zkCallback-774-thread-1-processing-n:127.0.0.1:38844_) [n:127.0.0.1:38844_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1312308 INFO (zkCallback-765-thread-1-processing-n:127.0.0.1:41989_) [n:127.0.0.1:41989_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1312769 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [n:127.0.0.1:38844_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_7B4EC0BE997D7FBF-001/shard-1-001/cores
[junit4] 2> 1312769 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [n:127.0.0.1:38844_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1313120 INFO (coreLoadExecutor-1415-thread-1-processing-n:127.0.0.1:38844_) [n:127.0.0.1:38844_ c:collection1 x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 1313137 INFO (OverseerStateUpdate-98549900914589700-127.0.0.1:41989_-n_0000000000) [n:127.0.0.1:41989_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 1314269 WARN (coreLoadExecutor-1415-thread-1-processing-n:127.0.0.1:38844_) [n:127.0.0.1:38844_ c:collection1 x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 1314270 INFO (coreLoadExecutor-1415-thread-1-processing-n:127.0.0.1:38844_) [n:127.0.0.1:38844_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.1
[junit4] 2> 1314356 INFO (coreLoadExecutor-1415-thread-1-processing-n:127.0.0.1:38844_) [n:127.0.0.1:38844_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1315176 INFO (coreLoadExecutor-1415-thread-1-processing-n:127.0.0.1:38844_) [n:127.0.0.1:38844_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 1315263 INFO (coreLoadExecutor-1415-thread-1-processing-n:127.0.0.1:38844_) [n:127.0.0.1:38844_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1, trusted=true
[junit4] 2> 1315276 INFO (coreLoadExecutor-1415-thread-1-processing-n:127.0.0.1:38844_) [n:127.0.0.1:38844_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://lucene2-us-west.apache.org:44350/solr_hdfs_home
[junit4] 2> 1315358 INFO (coreLoadExecutor-1415-thread-1-processing-n:127.0.0.1:38844_) [n:127.0.0.1:38844_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 1315358 INFO (coreLoadExecutor-1415-thread-1-processing-n:127.0.0.1:38844_) [n:127.0.0.1:38844_ c:collection1 x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1315358 INFO (coreLoadExecutor-1415-thread-1-processing-n:127.0.0.1:38844_) [n:127.0.0.1:38844_ c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_7B4EC0BE997D7FBF-001/shard-1-001/cores/collection1], dataDir=[hdfs://lucene2-us-west.apache.org:44350/solr_hdfs_home/collection1/core_node1/data/]
[junit4] 2> 1315358 INFO (coreLoadExecutor-1415-thread-1-processing-n:127.0.0.1:38844_) [n:127.0.0.1:38844_ c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@af328a6
[junit4] 2> 1315358 INFO (coreLoadExecutor-1415-thread-1-processing-n:127.0.0.1:38844_) [n:127.0.0.1:38844_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:44350/solr_hdfs_home/collection1/core_node1/data/snapshot_metadata
[junit4] 2> 1315423 INFO (coreLoadExecutor-1415-thread-1-processing-n:127.0.0.1:38844_) [n:127.0.0.1:38844_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1315423 INFO (coreLoadExecutor-1415-thread-1-processing-n:127.0.0.1:38844_) [n:127.0.0.1:38844_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 1315601 INFO (coreLoadExecutor-1415-thread-1-processing-n:127.0.0.1:38844_) [n:127.0.0.1:38844_ c:collection1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1315719 INFO (coreLoadExecutor-1415-thread-1-processing-n:127.0.0.1:38844_) [n:127.0.0.1:38844_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:44350/solr_hdfs_home/collection1/core_node1/data
[junit4] 2> 1315881 INFO (coreLoadExecutor-1415-thread-1-processing-n:127.0.0.1:38844_) [n:127.0.0.1:38844_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:44350/solr_hdfs_home/collection1/core_node1/data/index
[junit4] 2> 1315899 INFO (coreLoadExecutor-1415-thread-1-processing-n:127.0.0.1:38844_) [n:127.0.0.1:38844_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1315899 INFO (coreLoadExecutor-1415-thread-1-processing-n:127.0.0.1:38844_) [n:127.0.0.1:38844_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 1315962 INFO (coreLoadExecutor-1415-thread-1-processing-n:127.0.0.1:38844_) [n:127.0.0.1:38844_ c:collection1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1315963 INFO (coreLoadExecutor-1415-thread-1-processing-n:127.0.0.1:38844_) [n:127.0.0.1:38844_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@606acc3d
[junit4] 2> 1316438 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:34736 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-5c673ec8-4a98-40f2-a0ec-f0690acbd68a:NORMAL:127.0.0.1:35179|RBW], ReplicaUC[[DISK]DS-918d05a7-c681-4c5b-a0ef-137bd545f87a:NORMAL:127.0.0.1:34736|RBW]]} size 0
[junit4] 2> 1316503 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:35179 is added to blk_1073741826_1002 size 71
[junit4] 2> 1316572 WARN (coreLoadExecutor-1415-thread-1-processing-n:127.0.0.1:38844_) [n:127.0.0.1:38844_ c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1316885 INFO (coreLoadExecutor-1415-thread-1-processing-n:127.0.0.1:38844_) [n:127.0.0.1:38844_ c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 1316885 INFO (coreLoadExecutor-1415-thread-1-processing-n:127.0.0.1:38844_) [n:127.0.0.1:38844_ c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1316885 INFO (coreLoadExecutor-1415-thread-1-processing-n:127.0.0.1:38844_) [n:127.0.0.1:38844_ c:collection1 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 1317036 INFO (coreLoadExecutor-1415-thread-1-processing-n:127.0.0.1:38844_) [n:127.0.0.1:38844_ c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1317036 INFO (coreLoadExecutor-1415-thread-1-processing-n:127.0.0.1:38844_) [n:127.0.0.1:38844_ c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1317149 INFO (coreLoadExecutor-1415-thread-1-processing-n:127.0.0.1:38844_) [n:127.0.0.1:38844_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=9, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6603457310830974]
[junit4] 2> 1317504 INFO (coreLoadExecutor-1415-thread-1-processing-n:127.0.0.1:38844_) [n:127.0.0.1:38844_ c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2dda3e32[collection1] main]
[junit4] 2> 1317505 INFO (coreLoadExecutor-1415-thread-1-processing-n:127.0.0.1:38844_) [n:127.0.0.1:38844_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1317505 INFO (coreLoadExecutor-1415-thread-1-processing-n:127.0.0.1:38844_) [n:127.0.0.1:38844_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1317505 INFO (coreLoadExecutor-1415-thread-1-processing-n:127.0.0.1:38844_) [n:127.0.0.1:38844_ c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1317512 INFO (searcherExecutor-1416-thread-1-processing-n:127.0.0.1:38844_ x:collection1 c:collection1) [n:127.0.0.1:38844_ c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2dda3e32[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1317573 INFO (coreLoadExecutor-1415-thread-1-processing-n:127.0.0.1:38844_) [n:127.0.0.1:38844_ c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1576798434045722624
[junit4] 2> 1317616 INFO (coreZkRegister-1410-thread-1-processing-n:127.0.0.1:38844_ x:collection1 c:collection1) [n:127.0.0.1:38844_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1317616 INFO (coreZkRegister-1410-thread-1-processing-n:127.0.0.1:38844_ x:collection1 c:collection1) [n:127.0.0.1:38844_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1317616 INFO (coreZkRegister-1410-thread-1-processing-n:127.0.0.1:38844_ x:collection1 c:collection1) [n:127.0.0.1:38844_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:38844/collection1/
[junit4] 2> 1317630 INFO (coreZkRegister-1410-thread-1-processing-n:127.0.0.1:38844_ x:collection1 c:collection1) [n:127.0.0.1:38844_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1317630 INFO (coreZkRegister-1410-thread-1-processing-n:127.0.0.1:38844_ x:collection1 c:collection1) [n:127.0.0.1:38844_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:38844/collection1/ has no replicas
[junit4] 2> 1317630 INFO (coreZkRegister-1410-thread-1-processing-n:127.0.0.1:38844_ x:collection1 c:collection1) [n:127.0.0.1:38844_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 1317645 INFO (coreZkRegister-1410-thread-1-processing-n:127.0.0.1:38844_ x:collection1 c:collection1) [n:127.0.0.1:38844_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:38844/collection1/ shard2
[junit4] 2> 1317807 INFO (coreZkRegister-1410-thread-1-processing-n:127.0.0.1:38844_ x:collection1 c:collection1) [n:127.0.0.1:38844_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1318269 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_7B4EC0BE997D7FBF-001/shard-2-001/cores/collection1
[junit4] 2> 1318269 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_7B4EC0BE997D7FBF-001/shard-2-001
[junit4] 2> 1318270 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1318304 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7aa85485{/,null,AVAILABLE}
[junit4] 2> 1318304 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@75fb7ba1{HTTP/1.1,[http/1.1]}{127.0.0.1:40776}
[junit4] 2> 1318304 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.e.j.s.Server Started @1331127ms
[junit4] 2> 1318304 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://lucene2-us-west.apache.org:44350/hdfs__lucene2-us-west.apache.org_44350__home_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-6.6_checkout_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.HdfsUnloadDistributedZkTest_7B4EC0BE997D7FBF-001_tempDir-002_jetty2, solrconfig=solrconfig.xml, hostContext=/, hostPort=40776, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_7B4EC0BE997D7FBF-001/shard-2-001/cores}
[junit4] 2> 1318304 ERROR (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1318316 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.6.1
[junit4] 2> 1318316 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1318316 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1318316 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-26T12:56:09.442Z
[junit4] 2> 1318353 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1318353 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_7B4EC0BE997D7FBF-001/shard-2-001/solr.xml
[junit4] 2> 1318405 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 1318405 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35961/solr
[junit4] 2> 1318521 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [n:127.0.0.1:40776_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1318551 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [n:127.0.0.1:40776_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1318552 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [n:127.0.0.1:40776_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40776_
[junit4] 2> 1318554 INFO (zkCallback-769-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1318583 INFO (zkCallback-774-thread-1-processing-n:127.0.0.1:38844_) [n:127.0.0.1:38844_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1318583 INFO (zkCallback-765-thread-2-processing-n:127.0.0.1:41989_) [n:127.0.0.1:41989_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1318632 INFO (zkCallback-780-thread-1-processing-n:127.0.0.1:40776_) [n:127.0.0.1:40776_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1318945 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [n:127.0.0.1:40776_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_7B4EC0BE997D7FBF-001/shard-2-001/cores
[junit4] 2> 1318945 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[7B4EC0BE997D7FBF]) [n:127.0.0.1:40776_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1318973 INFO (coreLoadExecutor-1426-thread-1-processing-n:127.0.0.1:40776_) [n:127.0.0.1:40776_ c:collection1 x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 1319038 INFO (OverseerStateUpdate-98549900914589700-127.0.0.1:41989_-n_0000000000) [n:127.0.0.1:41989_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 1320148 WARN (coreLoadExecutor-1426-thread-1-processing-n:127.0.0.1:40776_) [n:127.0.0.1:40776_ c:collection1 x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 1320149 INFO (coreLoadExecutor-1426-thread-1-processing-n:127.0.0.1:40776_) [n:127.0.0.1:40776_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.1
[junit4] 2> 1320287 INFO (coreLoadExecutor-1426-thread-1-processing-n:127.0.0.1:40776_) [n:127.0.0.1:40776_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1321250 INFO (coreLoadExecutor-1426-thread-1-processing-n:127.0.0.1:40776_) [n:127.0.0.1:40776_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 1321303 INFO (coreLoadExecutor-1426-thread-1-processing-n:127.0.0.1:40776_) [n:127.0.0.1:40776_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1, trusted=true
[junit4] 2> 1321304 INFO (coreLoadExecutor-1426-thread-1-processing-n:127.0.0.1:40776_) [n:127.0.0.1:40776_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://lucene2-us-west.apache.org:44350/solr_hdfs_home
[junit4] 2> 1321304 INFO (coreLoadExecutor-1426-thread-1-processing-n:127.0.0.1:40776_) [n:127.0.0.1:40776_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 1321304 INFO (coreLoadExecutor-1426-thread-1-processing-n:127.0.0.1:40776_) [n:127.0.0.1:40776_ c:collection1 x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1321304 INFO (coreLoadExecutor-1426-thread-1-processing-n:127.0.0.1:40776_) [n:127.0.0.1:40776_ c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_7B4EC0BE997D7FBF-001/shard-2-001/cores/collection1], dataDir=[hdfs://lucene2-us-west.apache.org:44350/solr_hdfs_home/collection1/core_node2/data/]
[junit4] 2> 1321304 INFO (coreLoadExecutor-1426-thread-1-processing-n:127.0.0.1:40776_) [n:127.0.0.1:40776_ c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@af328a6
[junit4] 2> 1321304 INFO (coreLoadExecutor-1426-thread-1-processing-n:127.0.0.1:40776_) [n:127.0.0.1:40776_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:44350/solr_hdfs_home/collection1/core_node2/data/snapshot_metadata
[junit4] 2> 1321369 INFO (coreLoadExecutor-1426-thread-1-processing-n:127.0.0.1:40776_) [n:127.0.0.1:40776_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1321369 INFO (coreLoadExecutor-1426-thread-1-processing-n:127.0.0.1:40776_) [n:127.0.0.1:40776_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 1321409 INFO (coreLoadExecutor-1426-thread-1-processing-n:127.0.0.1:40776_) [n:127.0.0.1:40776_ c:collection1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1321410 INFO (coreLoadExecutor-1426-thread-1-processing-n:127.0.0.1:40776_) [n:127.0.0.1:40776_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:44350/solr_hdfs_home/collection1/core_node2/data
[junit4] 2> 1321608 INFO (coreLoadExecutor-1426-thread-1-processing-n:127.0.0.1:40776_) [n:127.0.0.1:40776_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:44350/solr_hdfs_home/collection1/core_node2/data/index
[junit4] 2> 1321614 INFO (coreLoadExecutor-1426-thread-1-processing-n:127.0.0.1:40776_) [n:127.0.0.1:40776_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1321614 INFO (coreLoadExecutor-1426-thread-1-processing-n:127.0.0.1:40776_) [n:127.0.0.1:40776_ c:collection1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 1321720 INFO (coreLoadExecutor-1426-thread-1-processing-n:127.0.0.1:40776_) [n:127.0.0.1:40776_ c:collection1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1321720 INFO (coreLoadExecutor-1426-thread-1-processing-n:127.0.0.1:40776_) [n:127.0.0.1:40776_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@6098c923
[junit4] 2> 1322099 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:35179 is added to blk_1073741827_1003{UCState=COMMITTED, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-918d05a7-c681-4c5b-a0ef-137bd545f87a:NORMAL:127.0.0.1:34736|RBW], ReplicaUC[[DISK]DS-5c673ec8-4a98-40f2-a0ec-f0690acbd68a:NORMAL:127.0.0.1:35179|RBW]]} size 71
[junit4] 2> 1322100 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:34736 is added to blk_1073741827_1003 size 71
[junit4] 2> 1322634 WARN (coreLoadExecutor-1426-thread-1-processing-n:127.0.0.1:40776_) [n:127.0.0.1:40776_ c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1323317 INFO (coreLoadExecutor-1426-thread-1-processing-n:127.0.0.1:40776_) [n:127.0.0.1:40776_ c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 1323317 INFO (coreLoadExecutor-1426-thread-1-processing-n:127.0.0.1:40776_) [n:127.0.0.1:40776_ c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1323317 INFO (coreLoadExecutor-1426-thread-1-processing-n:127.0.0.1:40776_) [n:127.0.0.1:40776_ c:collection1 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 1323415 INFO (coreLoadExecutor-1426-thread-1-processing-n:127.0.0.1:40776_) [n:127.0.0.1:40776_ c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1323415 INFO (coreLoadExecutor-1426-thread-1-processing-n:127.0.0.1:40776_) [n:127.0.0.1:40776_ c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1323631 INFO (coreLoadExecutor-1426-thread-1-processing-n:127.0.0.1:40776_) [n:127.0.0.1:40776_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=9, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6603457310830974]
[junit4] 2> 1323793 INFO (coreLoadExecutor-1426-thread-1-processing-n:127.0.0.1:40776_) [n:127.0.0.1:40776_ c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@16ce982e[collection1] main]
[junit4] 2> 1323793 INFO (coreLoadExecutor-1426-thread-1-processing-n:127.0.0.1:40776_) [n:127.0.0.1:40776_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1323794 INFO (coreLoadExecutor-1426-thread-1-processing-n:127.0.0.1:40776_) [n:127.0.0.1:40776_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1323794 INFO (coreLoadExecutor-1426-thread-1-processing-n:127.0.0.1:40776_) [n:127.0.0.1:40776_ c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1323795 INFO (searcherExecutor-1427-thread-1-processing-n:127.0.0.1:40776_ x:collection1 c:collection1) [n:127.0.0.1:40776_ c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@16ce982e[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1323984 INFO (coreLoadExecutor-1426-thread-1-processing-n:127.0.0.1:40776_) [n:127.0.0.1:40776_ c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1576798440768143360
[junit4] 2> 1324030 INFO (coreZkRegister-1421-thread-1-processing-n:127.0.0.1:40776_ x:collection1 c:collection1) [n:127.0.0.1:40776_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1324030 INFO (coreZkRegister-1421-thread-1-processing-n:127.0.0.1:40776_ x:collection1 c:collection1) [n:127.0.0.1:40776_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1324030 INFO (coreZkRegister-1421-thread-1-processing-n:127.0.0.1:40776_ x:collection1 c:collection1) [n:127.0.0.1:40776_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:40776/collection1/
[junit4] 2> 1324030 INFO (coreZkRegister-1421-thread-1-p
[...truncated too long message...]
] 2> 8521605 WARN (357838185@qtp-1648518890-1 - Acceptor0 HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34534) [ ] o.a.h.h.HttpServer2 HttpServer Acceptor: isRunning is false. Rechecking.
[junit4] 2> 8521605 WARN (357838185@qtp-1648518890-1 - Acceptor0 HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34534) [ ] o.a.h.h.HttpServer2 HttpServer Acceptor: isRunning is false
[junit4] 2> 8521713 WARN (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_7B4EC0BE997D7FBF-001/tempDir-001/hdfsBaseDir/data/data3/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_7B4EC0BE997D7FBF-001/tempDir-001/hdfsBaseDir/data/data4/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:46741) [ ] o.a.h.h.s.d.IncrementalBlockReportManager IncrementalBlockReportManager interrupted
[junit4] 2> 8521725 WARN (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_7B4EC0BE997D7FBF-001/tempDir-001/hdfsBaseDir/data/data3/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_7B4EC0BE997D7FBF-001/tempDir-001/hdfsBaseDir/data/data4/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:46741) [ ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-1367118031-127.0.0.1-1503759097418 (Datanode Uuid 65550ac9-bfc3-4c6d-b1fc-5729100e827b) service to lucene2-us-west.apache.org/127.0.0.1:46741
[junit4] 2> 8521758 WARN (SUITE-StressHdfsTest-seed#[7B4EC0BE997D7FBF]-worker) [ ] o.a.h.h.s.d.DirectoryScanner DirectoryScanner: shutdown has been called
[junit4] 2> 8521951 INFO (SUITE-StressHdfsTest-seed#[7B4EC0BE997D7FBF]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
[junit4] 2> 8522052 WARN (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_7B4EC0BE997D7FBF-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_7B4EC0BE997D7FBF-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:46741) [ ] o.a.h.h.s.d.IncrementalBlockReportManager IncrementalBlockReportManager interrupted
[junit4] 2> 8522052 WARN (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_7B4EC0BE997D7FBF-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_7B4EC0BE997D7FBF-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:46741) [ ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-1367118031-127.0.0.1-1503759097418 (Datanode Uuid 8f5313e3-f54a-4032-9775-71d7abd5c659) service to lucene2-us-west.apache.org/127.0.0.1:46741
[junit4] 2> 8522162 INFO (SUITE-StressHdfsTest-seed#[7B4EC0BE997D7FBF]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@lucene2-us-west.apache.org:0
[junit4] 2> 8522290 INFO (SUITE-StressHdfsTest-seed#[7B4EC0BE997D7FBF]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.StressHdfsTest_7B4EC0BE997D7FBF-001
[junit4] 2> Aug 26, 2017 2:56:13 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 34 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): {multiDefault=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), id=PostingsFormat(name=Memory doPackFST= false), text=FSTOrd50, txt_t=FSTOrd50}, docValues:{range_facet_l_dv=DocValuesFormat(name=Lucene54), _version_=DocValuesFormat(name=Asserting), intDefault=DocValuesFormat(name=Asserting), range_facet_i_dv=DocValuesFormat(name=Asserting), intDvoDefault=DocValuesFormat(name=Memory), range_facet_l=DocValuesFormat(name=Asserting), timestamp=DocValuesFormat(name=Asserting)}, maxPointsInLeafNode=1613, maxMBSortInHeap=7.568252243090306, sim=RandomSimilarity(queryNorm=false,coord=yes): {}, locale=hu-HU, timezone=Europe/Amsterdam
[junit4] 2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=212908776,total=534773760
[junit4] 2> NOTE: All tests run in this JVM: [TestCloudSchemaless, TestExactStatsCache, TestStressRecovery, TestOmitPositions, TestFilteredDocIdSet, ConjunctionSolrSpellCheckerTest, DocValuesNotIndexedTest, TestManagedSchema, CdcrReplicationHandlerTest, TestFieldCacheVsDocValues, OverseerStatusTest, TestCloudJSONFacetJoinDomain, BaseCdcrDistributedZkTest, DistributedFacetExistsSmallTest, BadCopyFieldTest, TestClassicSimilarityFactory, DistributedFacetPivotSmallAdvancedTest, DefaultValueUpdateProcessorTest, JvmMetricsTest, TestDocBasedVersionConstraints, TestFunctionQuery, PolyFieldTest, RecoveryAfterSoftCommitTest, SolrMetricReporterTest, TestPushWriter, TestInfoStreamLogging, TestManagedResourceStorage, LeaderInitiatedRecoveryOnCommitTest, HdfsUnloadDistributedZkTest, ExplicitHLLTest, TestDelegationWithHadoopAuth, WrapperMergePolicyFactoryTest, DateFieldTest, HLLUtilTest, CoreAdminCreateDiscoverTest, TestBlendedInfixSuggestions, JsonValidatorTest, DisMaxRequestHandlerTest, DataDrivenBlockJoinTest, TestJoin, TestReplicationHandlerBackup, HdfsChaosMonkeySafeLeaderTest, TestSolrCoreProperties, TestStressUserVersions, ConfigureRecoveryStrategyTest, BasicDistributedZkTest, TestStressLucene, TestJmxIntegration, CollectionReloadTest, DeleteNodeTest, DistributedQueryComponentCustomSortTest, OverseerCollectionConfigSetProcessorTest, ReturnFieldsTest, TestSolrCoreParser, TestLeaderElectionWithEmptyReplica, TestIBSimilarityFactory, SolrSlf4jReporterTest, TestCrossCoreJoin, PeerSyncTest, TestMinMaxOnMultiValuedField, SynonymTokenizerTest, TestMacros, SolrCLIZkUtilsTest, AnalysisAfterCoreReloadTest, BasicZkTest, TestSolrCLIRunExample, TestRecoveryHdfs, ChaosMonkeyNothingIsSafeTest, TestMultiWordSynonyms, TestCodecSupport, TestDocTermOrds, SolrRequestParserTest, BigEndianAscendingWordDeserializerTest, TestPivotHelperCode, TestLazyCores, TestFaceting, AliasIntegrationTest, TestLocalFSCloudBackupRestore, PathHierarchyTokenizerFactoryTest, LeaderElectionIntegrationTest, TestSolrQueryResponse, TestBulkSchemaConcurrent, RegexBoostProcessorTest, TestJsonRequest, DeleteStatusTest, TestDistributedSearch, TestFastWriter, TestBinaryResponseWriter, TestComponentsName, TestPKIAuthenticationPlugin, SSLMigrationTest, TestSolr4Spatial, TestRandomCollapseQParserPlugin, ClusterStateTest, TestSubQueryTransformerCrossCore, NotRequiredUniqueKeyTest, TestJettySolrRunner, TestClusterStateMutator, SolrCoreTest, TestRealTimeGet, TestRandomFlRTGCloud, TestConfigSetsAPI, TestSolrDeletionPolicy2, TestNumericTerms32, RequiredFieldsTest, ScriptEngineTest, MergeStrategyTest, TestCoreAdminApis, TestZkAclsWithHadoopAuth, OutputWriterTest, TestBadConfig, XsltUpdateRequestHandlerTest, DistributedTermsComponentTest, TestCoreContainer, QueryElevationComponentTest, BadIndexSchemaTest, ConvertedLegacyTest, TestSort, DirectUpdateHandlerTest, HighlighterTest, ShowFileRequestHandlerTest, CurrencyFieldOpenExchangeTest, DistributedQueryElevationComponentTest, CurrencyFieldXmlFileTest, SolrIndexSplitterTest, SignatureUpdateProcessorFactoryTest, TestExtendedDismaxParser, SuggesterFSTTest, CoreAdminHandlerTest, TestFoldingMultitermQuery, SuggesterTSTTest, SuggesterTest, SpatialFilterTest, SuggesterWFSTTest, TestCSVLoader, TestPseudoReturnFields, FieldMutatingUpdateProcessorTest, TestAtomicUpdateErrorCases, TestWordDelimiterFilterFactory, QueryEqualityTest, SortByFunctionTest, TestRemoteStreaming, DocValuesMultiTest, DistanceFunctionTest, TestSolrDeletionPolicy1, DebugComponentTest, CacheHeaderTest, TestSurroundQueryParser, LukeRequestHandlerTest, StandardRequestHandlerTest, DirectSolrSpellCheckerTest, DocumentAnalysisRequestHandlerTest, TestQueryTypes, PrimitiveFieldTypeTest, FileBasedSpellCheckerTest, TermsComponentTest, XmlUpdateRequestHandlerTest, TestValueSourceCache, MoreLikeThisHandlerTest, FastVectorHighlighterTest, LoggingHandlerTest, SolrPluginUtilsTest, TestCollationField, TestCSVResponseWriter, MBeansHandlerTest, TestAnalyzedSuggestions, UniqFieldsUpdateProcessorFactoryTest, JsonLoaderTest, TestPHPSerializedResponseWriter, PingRequestHandlerTest, CSVRequestHandlerTest, TestLFUCache, ResourceLoaderTest, PluginInfoTest, DateMathParserTest, TestDocumentBuilder, SliceStateTest, UUIDFieldTest, TestRTGBase, DistributedIntervalFacetingTest, DistribJoinFromCollectionTest, MigrateRouteKeyTest, MultiThreadedOCPTest, SimpleCollectionCreateDeleteTest, SolrCloudExampleTest, SolrXmlInZkTest, TestAuthenticationFramework, TestCloudRecovery, TestConfigSetsAPIExclusivity, TestConfigSetsAPIZkFailure, TestCryptoKeys, TestExclusionRuleCollectionAccess, TestLockTree, TestMiniSolrCloudCluster, TestRequestForwarding, TestRequestStatusCollectionAPI, TestSolrCloudWithKerberosAlt, TestStressLiveNodes, HdfsRecoverLeaseTest, HdfsSyncSliceTest, HdfsTlogReplayBufferedWhileIndexingTest, StressHdfsTest]
[junit4] Completed [580/713 (5!)] on J0 in 284.87s, 1 test, 1 error <<< FAILURES!
[...truncated 43872 lines...]
[JENKINS] Lucene-Solr-NightlyTests-6.6 - Build # 26 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.6/26/
9 tests failed.
FAILED: org.apache.lucene.search.TestInetAddressRangeQueries.testRandomBig
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([F8BD8548E57E4F77]:0)
FAILED: junit.framework.TestSuite.org.apache.lucene.search.TestInetAddressRangeQueries
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([F8BD8548E57E4F77]:0)
FAILED: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test
Error Message:
Error from server at http://127.0.0.1:34779/collection1: Cannot talk to ZooKeeper - Updates are disabled.
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:34779/collection1: Cannot talk to ZooKeeper - Updates are disabled.
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:612)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:160)
at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:484)
at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:463)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.commit(AbstractFullDistribZkTestBase.java:1529)
at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testIndexingBatchPerRequestWithHttpSolrClient(FullSolrCloudDistribCmdsTest.java:664)
at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test(FullSolrCloudDistribCmdsTest.java:152)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test
Error Message:
Expected 2 of 3 replicas to be active but only found 1; [core_node3:{"core":"c8n_1x3_lf_shard1_replica2","base_url":"http://127.0.0.1:34918/vw","node_name":"127.0.0.1:34918_vw","state":"active","leader":"true"}]; clusterState: DocCollection(c8n_1x3_lf//clusterstate.json/31)={ "replicationFactor":"3", "shards":{"shard1":{ "range":"80000000-7fffffff", "state":"active", "replicas":{ "core_node1":{ "core":"c8n_1x3_lf_shard1_replica1", "base_url":"http://127.0.0.1:35037/vw", "node_name":"127.0.0.1:35037_vw", "state":"down"}, "core_node2":{ "state":"down", "base_url":"http://127.0.0.1:46527/vw", "core":"c8n_1x3_lf_shard1_replica3", "node_name":"127.0.0.1:46527_vw"}, "core_node3":{ "core":"c8n_1x3_lf_shard1_replica2", "base_url":"http://127.0.0.1:34918/vw", "node_name":"127.0.0.1:34918_vw", "state":"active", "leader":"true"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"false"}
Stack Trace:
java.lang.AssertionError: Expected 2 of 3 replicas to be active but only found 1; [core_node3:{"core":"c8n_1x3_lf_shard1_replica2","base_url":"http://127.0.0.1:34918/vw","node_name":"127.0.0.1:34918_vw","state":"active","leader":"true"}]; clusterState: DocCollection(c8n_1x3_lf//clusterstate.json/31)={
"replicationFactor":"3",
"shards":{"shard1":{
"range":"80000000-7fffffff",
"state":"active",
"replicas":{
"core_node1":{
"core":"c8n_1x3_lf_shard1_replica1",
"base_url":"http://127.0.0.1:35037/vw",
"node_name":"127.0.0.1:35037_vw",
"state":"down"},
"core_node2":{
"state":"down",
"base_url":"http://127.0.0.1:46527/vw",
"core":"c8n_1x3_lf_shard1_replica3",
"node_name":"127.0.0.1:46527_vw"},
"core_node3":{
"core":"c8n_1x3_lf_shard1_replica2",
"base_url":"http://127.0.0.1:34918/vw",
"node_name":"127.0.0.1:34918_vw",
"state":"active",
"leader":"true"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false"}
at __randomizedtesting.SeedInfo.seed([37B7808343C9655C:BFE3BF59ED3508A4]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.testRf3WithLeaderFailover(LeaderFailoverAfterPartitionTest.java:168)
at org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test(LeaderFailoverAfterPartitionTest.java:55)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.hdfs.StressHdfsTest.test
Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:40590
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:40590
at __randomizedtesting.SeedInfo.seed([37B7808343C9655C:BFE3BF59ED3508A4]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:623)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:447)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:388)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1383)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1134)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:1073)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at org.apache.solr.cloud.hdfs.StressHdfsTest.createAndDeleteCollection(StressHdfsTest.java:220)
at org.apache.solr.cloud.hdfs.StressHdfsTest.test(StressHdfsTest.java:103)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:515)
... 51 more
FAILED: org.apache.solr.client.solrj.TestSolrJErrorHandling.testWithXml
Error Message:
expected:<747> but was:<737>
Stack Trace:
java.lang.AssertionError: expected:<747> but was:<737>
at __randomizedtesting.SeedInfo.seed([D635EEA6D10DC868:7DD75750530BD105]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.client.solrj.TestSolrJErrorHandling.doThreads(TestSolrJErrorHandling.java:185)
at org.apache.solr.client.solrj.TestSolrJErrorHandling.doIt(TestSolrJErrorHandling.java:200)
at org.apache.solr.client.solrj.TestSolrJErrorHandling.testWithXml(TestSolrJErrorHandling.java:110)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([BBAAF2BDDFDCCC85]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.metrics.reporters.SolrJmxReporterCloudTest
Error Message:
Clean up static fields (in @AfterClass?), your test seems to hang on to approximately 19,292,328 bytes (threshold is 10,485,760). Field reference sizes (counted individually): - 19,291,824 bytes, private static javax.management.MBeanServer org.apache.solr.metrics.reporters.SolrJmxReporterCloudTest.mBeanServer - 192 bytes, public static org.junit.rules.TestRule org.apache.solr.SolrTestCaseJ4.solrClassRules - 128 bytes, private static java.lang.String org.apache.solr.SolrTestCaseJ4.factoryProp - 112 bytes, private static java.lang.String org.apache.solr.metrics.reporters.SolrJmxReporterCloudTest.COLLECTION - 64 bytes, private static java.lang.String org.apache.solr.SolrTestCaseJ4.coreName - 48 bytes, private static java.util.Map org.apache.solr.SolrTestCaseJ4.savedClassLogLevels
Stack Trace:
junit.framework.AssertionFailedError: Clean up static fields (in @AfterClass?), your test seems to hang on to approximately 19,292,328 bytes (threshold is 10,485,760). Field reference sizes (counted individually):
- 19,291,824 bytes, private static javax.management.MBeanServer org.apache.solr.metrics.reporters.SolrJmxReporterCloudTest.mBeanServer
- 192 bytes, public static org.junit.rules.TestRule org.apache.solr.SolrTestCaseJ4.solrClassRules
- 128 bytes, private static java.lang.String org.apache.solr.SolrTestCaseJ4.factoryProp
- 112 bytes, private static java.lang.String org.apache.solr.metrics.reporters.SolrJmxReporterCloudTest.COLLECTION
- 64 bytes, private static java.lang.String org.apache.solr.SolrTestCaseJ4.coreName
- 48 bytes, private static java.util.Map org.apache.solr.SolrTestCaseJ4.savedClassLogLevels
at __randomizedtesting.SeedInfo.seed([37B7808343C9655C]:0)
at com.carrotsearch.randomizedtesting.rules.StaticFieldsInvariantRule$1.afterAlways(StaticFieldsInvariantRule.java:170)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.lucene.spatial3d.TestGeo3DPoint
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([BBAAF2BDDFDCCC85]:0)
Build Log:
[...truncated 8879 lines...]
[junit4] Suite: org.apache.lucene.search.TestInetAddressRangeQueries
[junit4] 2> avg 23, 2017 2:12:32 AM com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
[junit4] 2> WARNING: Suite execution timed out: org.apache.lucene.search.TestInetAddressRangeQueries
[junit4] 2> 1) Thread[id=9, name=JUnit4-serializer-daemon, state=TIMED_WAITING, group=main]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:50)
[junit4] 2> 2) Thread[id=94, name=SUITE-TestInetAddressRangeQueries-seed#[F8BD8548E57E4F77], state=RUNNABLE, group=TGRP-TestInetAddressRangeQueries]
[junit4] 2> at java.lang.Thread.getStackTrace(Thread.java:1559)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:696)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:693)
[junit4] 2> at java.security.AccessController.doPrivileged(Native Method)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getStackTrace(ThreadLeakControl.java:693)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getThreadsWithTraces(ThreadLeakControl.java:709)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:689)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:65)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:415)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:678)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:140)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:598)
[junit4] 2> 3) Thread[id=1, name=main, state=WAITING, group=main]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Thread.join(Thread.java:1252)
[junit4] 2> at java.lang.Thread.join(Thread.java:1326)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:608)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:457)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:244)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:355)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:13)
[junit4] 2> 4) Thread[id=95, name=TEST-TestInetAddressRangeQueries.testRandomBig-seed#[F8BD8548E57E4F77], state=RUNNABLE, group=TGRP-TestInetAddressRangeQueries]
[junit4] 2> at java.lang.Throwable.fillInStackTrace(Native Method)
[junit4] 2> at java.lang.Throwable.fillInStackTrace(Throwable.java:783)
[junit4] 2> at java.lang.Throwable.<init>(Throwable.java:250)
[junit4] 2> at java.lang.Exception.<init>(Exception.java:54)
[junit4] 2> at java.lang.RuntimeException.<init>(RuntimeException.java:51)
[junit4] 2> at java.lang.UnsupportedOperationException.<init>(UnsupportedOperationException.java:42)
[junit4] 2> at org.apache.lucene.codecs.compressing.CompressingTermVectorsReader$TVTermsEnum.ord(CompressingTermVectorsReader.java:918)
[junit4] 2> at org.apache.lucene.index.CheckIndex.checkFields(CheckIndex.java:1384)
[junit4] 2> at org.apache.lucene.index.CheckIndex.testTermVectors(CheckIndex.java:2375)
[junit4] 2> at org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:751)
[junit4] 2> at org.apache.lucene.util.TestUtil.checkIndex(TestUtil.java:300)
[junit4] 2> at org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:865)
[junit4] 2> at org.apache.lucene.util.IOUtils.close(IOUtils.java:89)
[junit4] 2> at org.apache.lucene.util.IOUtils.close(IOUtils.java:76)
[junit4] 2> at org.apache.lucene.search.BaseRangeFieldQueryTestCase.verify(BaseRangeFieldQueryTestCase.java:292)
[junit4] 2> at org.apache.lucene.search.BaseRangeFieldQueryTestCase.doTestRandom(BaseRangeFieldQueryTestCase.java:160)
[junit4] 2> at org.apache.lucene.search.BaseRangeFieldQueryTestCase.testRandomBig(BaseRangeFieldQueryTestCase.java:75)
[junit4] 2> at org.apache.lucene.search.TestInetAddressRangeQueries.testRandomBig(TestInetAddressRangeQueries.java:81)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestInetAddressRangeQueries -Dtests.method=testRandomBig -Dtests.seed=F8BD8548E57E4F77 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/test-data/enwiki.random.lines.txt -Dtests.locale=sr-Latn-RS -Dtests.timezone=Asia/Macau -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] ERROR 7198s J1 | TestInetAddressRangeQueries.testRandomBig <<<
[junit4] > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
[junit4] > at __randomizedtesting.SeedInfo.seed([F8BD8548E57E4F77]:0)
[junit4] 2> avg 23, 2017 2:12:32 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> avg 23, 2017 2:12:53 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.lucene.search.TestInetAddressRangeQueries:
[junit4] 2> 1) Thread[id=95, name=TEST-TestInetAddressRangeQueries.testRandomBig-seed#[F8BD8548E57E4F77], state=RUNNABLE, group=TGRP-TestInetAddressRangeQueries]
[junit4] 2> at org.apache.lucene.store.MockIndexInputWrapper.readByte(MockIndexInputWrapper.java:140)
[junit4] 2> at org.apache.lucene.codecs.compressing.LZ4.decompress(LZ4.java:110)
[junit4] 2> at org.apache.lucene.codecs.compressing.CompressionMode$4.decompress(CompressionMode.java:138)
[junit4] 2> at org.apache.lucene.codecs.compressing.CompressingTermVectorsReader.get(CompressingTermVectorsReader.java:560)
[junit4] 2> at org.apache.lucene.index.CheckIndex.testTermVectors(CheckIndex.java:2368)
[junit4] 2> at org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:751)
[junit4] 2> at org.apache.lucene.util.TestUtil.checkIndex(TestUtil.java:300)
[junit4] 2> at org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:865)
[junit4] 2> at org.apache.lucene.util.IOUtils.close(IOUtils.java:89)
[junit4] 2> at org.apache.lucene.util.IOUtils.close(IOUtils.java:76)
[junit4] 2> at org.apache.lucene.search.BaseRangeFieldQueryTestCase.verify(BaseRangeFieldQueryTestCase.java:292)
[junit4] 2> at org.apache.lucene.search.BaseRangeFieldQueryTestCase.doTestRandom(BaseRangeFieldQueryTestCase.java:160)
[junit4] 2> at org.apache.lucene.search.BaseRangeFieldQueryTestCase.testRandomBig(BaseRangeFieldQueryTestCase.java:75)
[junit4] 2> at org.apache.lucene.search.TestInetAddressRangeQueries.testRandomBig(TestInetAddressRangeQueries.java:81)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> avg 23, 2017 2:12:53 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=95, name=TEST-TestInetAddressRangeQueries.testRandomBig-seed#[F8BD8548E57E4F77], state=RUNNABLE, group=TGRP-TestInetAddressRangeQueries]
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/lucene/build/misc/test/J1/temp/lucene.search.TestInetAddressRangeQueries_F8BD8548E57E4F77-001
[junit4] 2> Aug 22, 2017 6:12:53 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: All leaked threads terminated.
[junit4] 2> NOTE: test params are: codec=Lucene62, sim=RandomSimilarity(queryNorm=false,coord=no): {}, locale=sr-Latn-RS, timezone=Asia/Macau
[junit4] 2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=126919832,total=477626368
[junit4] 2> NOTE: All tests run in this JVM: [TestFieldCacheReopen, TestFieldCacheSortRandom, TestLazyDocument, TestLegacyFieldCache, TestRAFDirectory, TestFSTsMisc, TestIndexSplitter, TestPKIndexSplitter, TestHighFreqTerms, SweetSpotSimilarityTest, TestDocValuesStatsCollector, TestInetAddressRangeQueries]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestInetAddressRangeQueries -Dtests.seed=F8BD8548E57E4F77 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/test-data/enwiki.random.lines.txt -Dtests.locale=sr-Latn-RS -Dtests.timezone=Asia/Macau -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] ERROR 0.00s J1 | TestInetAddressRangeQueries (suite) <<<
[junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4] > at __randomizedtesting.SeedInfo.seed([F8BD8548E57E4F77]:0)
[junit4] Completed [26/26 (1!)] on J1 in 7223.10s, 2 tests, 2 errors <<< FAILURES!
[...truncated 1658 lines...]
[junit4] Suite: org.apache.lucene.spatial3d.TestGeo3DPoint
[junit4] 2> aug 22, 2017 11:59:24 AM com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
[junit4] 2> WARNING: Suite execution timed out: org.apache.lucene.spatial3d.TestGeo3DPoint
[junit4] 2> 1) Thread[id=12, name=TEST-TestGeo3DPoint.testRandomBig-seed#[BBAAF2BDDFDCCC85], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at sun.nio.ch.NativeThread.current(Native Method)
[junit4] 2> at sun.nio.ch.NativeThreadSet.add(NativeThreadSet.java:46)
[junit4] 2> at sun.nio.ch.FileChannelImpl.readInternal(FileChannelImpl.java:737)
[junit4] 2> at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:727)
[junit4] 2> at org.apache.lucene.mockfile.FilterFileChannel.read(FilterFileChannel.java:111)
[junit4] 2> at org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.readInternal(NIOFSDirectory.java:179)
[junit4] 2> at org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:342)
[junit4] 2> at org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:54)
[junit4] 2> at org.apache.lucene.store.DataInput.readInt(DataInput.java:102)
[junit4] 2> at org.apache.lucene.store.BufferedIndexInput.readInt(BufferedIndexInput.java:183)
[junit4] 2> at org.apache.lucene.store.MockIndexInputWrapper.readInt(MockIndexInputWrapper.java:165)
[junit4] 2> at org.apache.lucene.store.IndexInput$1.readInt(IndexInput.java:144)
[junit4] 2> at org.apache.lucene.util.packed.DirectReader$DirectPackedReader20.get(DirectReader.java:206)
[junit4] 2> at org.apache.lucene.codecs.lucene54.Lucene54DocValuesProducer$2.get(Lucene54DocValuesProducer.java:501)
[junit4] 2> at org.apache.lucene.util.LongValues.get(LongValues.java:45)
[junit4] 2> at org.apache.lucene.index.MultiDocValues$2.get(MultiDocValues.java:141)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.verify(TestGeo3DPoint.java:839)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.doTestRandom(TestGeo3DPoint.java:548)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig(TestGeo3DPoint.java:480)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 2) Thread[id=1, name=main, state=WAITING, group=main]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Thread.join(Thread.java:1252)
[junit4] 2> at java.lang.Thread.join(Thread.java:1326)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:608)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:457)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:244)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:355)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:13)
[junit4] 2> 3) Thread[id=11, name=SUITE-TestGeo3DPoint-seed#[BBAAF2BDDFDCCC85], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at java.lang.Thread.getStackTrace(Thread.java:1559)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:696)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:693)
[junit4] 2> at java.security.AccessController.doPrivileged(Native Method)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getStackTrace(ThreadLeakControl.java:693)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getThreadsWithTraces(ThreadLeakControl.java:709)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:689)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:65)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:415)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:678)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:140)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:598)
[junit4] 2> 4) Thread[id=9, name=JUnit4-serializer-daemon, state=TIMED_WAITING, group=main]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:50)
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestGeo3DPoint -Dtests.method=testRandomBig -Dtests.seed=BBAAF2BDDFDCCC85 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/test-data/enwiki.random.lines.txt -Dtests.locale=nl -Dtests.timezone=Pacific/Gambier -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] ERROR 7198s J1 | TestGeo3DPoint.testRandomBig <<<
[junit4] > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
[junit4] > at __randomizedtesting.SeedInfo.seed([BBAAF2BDDFDCCC85]:0)
[junit4] 2> aug 22, 2017 11:59:24 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> aug 22, 2017 11:59:44 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.lucene.spatial3d.TestGeo3DPoint:
[junit4] 2> 1) Thread[id=12, name=TEST-TestGeo3DPoint.testRandomBig-seed#[BBAAF2BDDFDCCC85], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at org.apache.lucene.spatial3d.geom.Plane.findIntersectionBounds(Plane.java:1097)
[junit4] 2> at org.apache.lucene.spatial3d.geom.Plane.recordBounds(Plane.java:1214)
[junit4] 2> at org.apache.lucene.spatial3d.geom.XYZBounds.addIntersection(XYZBounds.java:262)
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoConvexPolygon.getBounds(GeoConvexPolygon.java:425)
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoCompositeMembershipShape.getBounds(GeoCompositeMembershipShape.java:75)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.verify(TestGeo3DPoint.java:845)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.doTestRandom(TestGeo3DPoint.java:548)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig(TestGeo3DPoint.java:480)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> aug 22, 2017 11:59:44 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=12, name=TEST-TestGeo3DPoint.testRandomBig-seed#[BBAAF2BDDFDCCC85], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/lucene/build/spatial3d/test/J1/temp/lucene.spatial3d.TestGeo3DPoint_BBAAF2BDDFDCCC85-001
[junit4] 2> Aug 22, 2017 8:59:44 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: All leaked threads terminated.
[junit4] 2> NOTE: test params are: codec=HighCompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=HIGH_COMPRESSION, chunkSize=2, maxDocsPerChunk=818, blockSize=333), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=HIGH_COMPRESSION, chunkSize=2, blockSize=333)), sim=RandomSimilarity(queryNorm=true,coord=crazy): {}, locale=nl, timezone=Pacific/Gambier
[junit4] 2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=98270280,total=490733568
[junit4] 2> NOTE: All tests run in this JVM: [TestGeo3DPoint]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestGeo3DPoint -Dtests.seed=BBAAF2BDDFDCCC85 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/test-data/enwiki.random.lines.txt -Dtests.locale=nl -Dtests.timezone=Pacific/Gambier -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] ERROR 0.00s J1 | TestGeo3DPoint (suite) <<<
[junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4] > at __randomizedtesting.SeedInfo.seed([BBAAF2BDDFDCCC85]:0)
[junit4] Completed [10/11 (1!)] on J1 in 7221.98s, 1 test, 2 errors <<< FAILURES!
[...truncated 1745 lines...]
[junit4] Suite: org.apache.solr.cloud.LeaderFailoverAfterPartitionTest
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderFailoverAfterPartitionTest_37B7808343C9655C-001/init-core-data-001
[junit4] 2> 297612 INFO (SUITE-LeaderFailoverAfterPartitionTest-seed#[37B7808343C9655C]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields
[junit4] 2> 297633 INFO (SUITE-LeaderFailoverAfterPartitionTest-seed#[37B7808343C9655C]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
[junit4] 2> 297633 INFO (SUITE-LeaderFailoverAfterPartitionTest-seed#[37B7808343C9655C]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /vw/
[junit4] 2> 297693 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 297702 INFO (Thread-238) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 297702 INFO (Thread-238) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 297718 ERROR (Thread-238) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 297811 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.a.s.c.ZkTestServer start zk server on port:34270
[junit4] 2> 298269 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 298270 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 298271 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 298272 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 298272 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 298273 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 298274 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 298274 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 298275 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 298297 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 298298 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 299237 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderFailoverAfterPartitionTest_37B7808343C9655C-001/control-001/cores/collection1
[junit4] 2> 299268 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 299403 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@12b30943{/vw,null,AVAILABLE}
[junit4] 2> 299405 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@58add6c8{HTTP/1.1,[http/1.1]}{127.0.0.1:38697}
[junit4] 2> 299405 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.e.j.s.Server Started @312074ms
[junit4] 2> 299405 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderFailoverAfterPartitionTest_37B7808343C9655C-001/tempDir-001/control/data, hostContext=/vw, hostPort=35037, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderFailoverAfterPartitionTest_37B7808343C9655C-001/control-001/cores}
[junit4] 2> 299406 ERROR (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 299406 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.6.1
[junit4] 2> 299439 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 299439 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 299439 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-22T21:12:31.689Z
[junit4] 2> 299505 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 299505 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderFailoverAfterPartitionTest_37B7808343C9655C-001/control-001/solr.xml
[junit4] 2> 299539 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 299552 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34270/solr
[junit4] 2> 299787 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [n:127.0.0.1:35037_vw ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 299787 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [n:127.0.0.1:35037_vw ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:35037_vw
[junit4] 2> 299817 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [n:127.0.0.1:35037_vw ] o.a.s.c.Overseer Overseer (id=98529204630454276-127.0.0.1:35037_vw-n_0000000000) starting
[junit4] 2> 299898 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [n:127.0.0.1:35037_vw ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35037_vw
[junit4] 2> 299899 INFO (zkCallback-307-thread-1-processing-n:127.0.0.1:35037_vw) [n:127.0.0.1:35037_vw ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 300373 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [n:127.0.0.1:35037_vw ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderFailoverAfterPartitionTest_37B7808343C9655C-001/control-001/cores
[junit4] 2> 300374 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [n:127.0.0.1:35037_vw ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 300459 INFO (coreLoadExecutor-456-thread-1-processing-n:127.0.0.1:35037_vw) [n:127.0.0.1:35037_vw c:control_collection x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 300460 INFO (OverseerStateUpdate-98529204630454276-127.0.0.1:35037_vw-n_0000000000) [n:127.0.0.1:35037_vw ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 300574 INFO (coreLoadExecutor-456-thread-1-processing-n:127.0.0.1:35037_vw) [n:127.0.0.1:35037_vw c:control_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.1
[junit4] 2> 300789 INFO (coreLoadExecutor-456-thread-1-processing-n:127.0.0.1:35037_vw) [n:127.0.0.1:35037_vw c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 301423 INFO (coreLoadExecutor-456-thread-1-processing-n:127.0.0.1:35037_vw) [n:127.0.0.1:35037_vw c:control_collection x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 301457 INFO (coreLoadExecutor-456-thread-1-processing-n:127.0.0.1:35037_vw) [n:127.0.0.1:35037_vw c:control_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection, trusted=true
[junit4] 2> 301457 INFO (coreLoadExecutor-456-thread-1-processing-n:127.0.0.1:35037_vw) [n:127.0.0.1:35037_vw c:control_collection x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 301457 INFO (coreLoadExecutor-456-thread-1-processing-n:127.0.0.1:35037_vw) [n:127.0.0.1:35037_vw c:control_collection x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderFailoverAfterPartitionTest_37B7808343C9655C-001/control-001/cores/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderFailoverAfterPartitionTest_37B7808343C9655C-001/control-001/cores/collection1/data/]
[junit4] 2> 301458 INFO (coreLoadExecutor-456-thread-1-processing-n:127.0.0.1:35037_vw) [n:127.0.0.1:35037_vw c:control_collection x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6a29347
[junit4] 2> 301459 INFO (coreLoadExecutor-456-thread-1-processing-n:127.0.0.1:35037_vw) [n:127.0.0.1:35037_vw c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=11, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.29639301721677114]
[junit4] 2> 301651 WARN (coreLoadExecutor-456-thread-1-processing-n:127.0.0.1:35037_vw) [n:127.0.0.1:35037_vw c:control_collection x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 301819 INFO (coreLoadExecutor-456-thread-1-processing-n:127.0.0.1:35037_vw) [n:127.0.0.1:35037_vw c:control_collection x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 301819 INFO (coreLoadExecutor-456-thread-1-processing-n:127.0.0.1:35037_vw) [n:127.0.0.1:35037_vw c:control_collection x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 301819 INFO (coreLoadExecutor-456-thread-1-processing-n:127.0.0.1:35037_vw) [n:127.0.0.1:35037_vw c:control_collection x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 301819 INFO (coreLoadExecutor-456-thread-1-processing-n:127.0.0.1:35037_vw) [n:127.0.0.1:35037_vw c:control_collection x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 301820 INFO (coreLoadExecutor-456-thread-1-processing-n:127.0.0.1:35037_vw) [n:127.0.0.1:35037_vw c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@32067512
[junit4] 2> 301820 INFO (coreLoadExecutor-456-thread-1-processing-n:127.0.0.1:35037_vw) [n:127.0.0.1:35037_vw c:control_collection x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@6832492f[collection1] main]
[junit4] 2> 301850 INFO (coreLoadExecutor-456-thread-1-processing-n:127.0.0.1:35037_vw) [n:127.0.0.1:35037_vw c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 301850 INFO (coreLoadExecutor-456-thread-1-processing-n:127.0.0.1:35037_vw) [n:127.0.0.1:35037_vw c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 301850 INFO (coreLoadExecutor-456-thread-1-processing-n:127.0.0.1:35037_vw) [n:127.0.0.1:35037_vw c:control_collection x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 301932 INFO (searcherExecutor-457-thread-1-processing-n:127.0.0.1:35037_vw x:collection1 c:control_collection) [n:127.0.0.1:35037_vw c:control_collection x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@6832492f[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 301932 INFO (coreLoadExecutor-456-thread-1-processing-n:127.0.0.1:35037_vw) [n:127.0.0.1:35037_vw c:control_collection x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1576467278522744832
[junit4] 2> 301953 INFO (coreZkRegister-449-thread-1-processing-n:127.0.0.1:35037_vw x:collection1 c:control_collection) [n:127.0.0.1:35037_vw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 301953 INFO (coreZkRegister-449-thread-1-processing-n:127.0.0.1:35037_vw x:collection1 c:control_collection) [n:127.0.0.1:35037_vw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 301953 INFO (coreZkRegister-449-thread-1-processing-n:127.0.0.1:35037_vw x:collection1 c:control_collection) [n:127.0.0.1:35037_vw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:35037/vw/collection1/
[junit4] 2> 301953 INFO (coreZkRegister-449-thread-1-processing-n:127.0.0.1:35037_vw x:collection1 c:control_collection) [n:127.0.0.1:35037_vw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 301953 INFO (coreZkRegister-449-thread-1-processing-n:127.0.0.1:35037_vw x:collection1 c:control_collection) [n:127.0.0.1:35037_vw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:35037/vw/collection1/ has no replicas
[junit4] 2> 301953 INFO (coreZkRegister-449-thread-1-processing-n:127.0.0.1:35037_vw x:collection1 c:control_collection) [n:127.0.0.1:35037_vw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 301973 INFO (coreZkRegister-449-thread-1-processing-n:127.0.0.1:35037_vw x:collection1 c:control_collection) [n:127.0.0.1:35037_vw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:35037/vw/collection1/ shard1
[junit4] 2> 302078 INFO (coreZkRegister-449-thread-1-processing-n:127.0.0.1:35037_vw x:collection1 c:control_collection) [n:127.0.0.1:35037_vw c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 302100 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 302100 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34270/solr ready
[junit4] 2> 302100 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 302528 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderFailoverAfterPartitionTest_37B7808343C9655C-001/shard-1-001/cores/collection1
[junit4] 2> 302541 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderFailoverAfterPartitionTest_37B7808343C9655C-001/shard-1-001
[junit4] 2> 302542 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 302558 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4bec60de{/vw,null,AVAILABLE}
[junit4] 2> 302558 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@580a7834{HTTP/1.1,[http/1.1]}{127.0.0.1:33354}
[junit4] 2> 302558 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.e.j.s.Server Started @315228ms
[junit4] 2> 302558 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderFailoverAfterPartitionTest_37B7808343C9655C-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/vw, hostPort=34918, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderFailoverAfterPartitionTest_37B7808343C9655C-001/shard-1-001/cores}
[junit4] 2> 302559 ERROR (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 302573 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.6.1
[junit4] 2> 302573 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 302573 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 302574 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-22T21:12:34.823Z
[junit4] 2> 302659 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 302659 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderFailoverAfterPartitionTest_37B7808343C9655C-001/shard-1-001/solr.xml
[junit4] 2> 302693 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 302695 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34270/solr
[junit4] 2> 302768 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [n:127.0.0.1:34918_vw ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 302796 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [n:127.0.0.1:34918_vw ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 302828 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [n:127.0.0.1:34918_vw ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34918_vw
[junit4] 2> 302841 INFO (zkCallback-311-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 302843 INFO (zkCallback-307-thread-2-processing-n:127.0.0.1:35037_vw) [n:127.0.0.1:35037_vw ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 302955 INFO (zkCallback-316-thread-1-processing-n:127.0.0.1:34918_vw) [n:127.0.0.1:34918_vw ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 303191 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [n:127.0.0.1:34918_vw ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderFailoverAfterPartitionTest_37B7808343C9655C-001/shard-1-001/cores
[junit4] 2> 303191 INFO (TEST-LeaderFailoverAfterPartitionTest.test-seed#[37B7808343C9655C]) [n:127.0.0.1:34918_vw ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 303204 INFO (coreLoadExecutor-467-thread-1-processing-n:127.0.0.1:34918_vw) [n:127.0.0.1:34918_vw c:collection1 x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 303240 INFO (OverseerStateUpdate-98529204630454276-127.0.0.1:35037_vw-n_0000000000) [n:127.0.0.1:35037_vw ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 304317 INFO (coreLoadExecutor-467-thread-1-proce
[...truncated too long message...]
jvm
[junit4] 2> 155028 INFO (SUITE-TestSolrJErrorHandling-seed#[D635EEA6D10DC868]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.jetty
[junit4] 2> 155051 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@9ba3480
[junit4] 2> 155052 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-54-thread-1]: now flush at close
[junit4] 2> 155052 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-54-thread-1]: start flush: applyAllDeletes=true
[junit4] 2> 155052 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-54-thread-1]: index before flush _7(6.6.1):C383/382:delGen=1 _8(6.6.1):C10/10:delGen=1
[junit4] 2> 155052 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [DW][coreCloseExecutor-54-thread-1]: startFullFlush
[junit4] 2> 155052 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [DW][coreCloseExecutor-54-thread-1]: coreCloseExecutor-54-thread-1 finishFullFlush success=true
[junit4] 2> 155052 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-54-thread-1]: apply all deletes during flush
[junit4] 2> 155052 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-54-thread-1]: now apply all deletes for all segments maxDoc=393
[junit4] 2> 155052 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [BD][coreCloseExecutor-54-thread-1]: applyDeletes: open segment readers took 0 msec
[junit4] 2> 155052 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [BD][coreCloseExecutor-54-thread-1]: applyDeletes: no segments; skipping
[junit4] 2> 155052 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [BD][coreCloseExecutor-54-thread-1]: prune sis=segments_7: _7(6.6.1):C383/382:delGen=1 _8(6.6.1):C10/10:delGen=1 minGen=34 packetCount=0
[junit4] 2> 155052 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [MS][coreCloseExecutor-54-thread-1]: updateMergeThreads ioThrottle=true targetMBPerSec=10240.0 MB/sec
[junit4] 2> 155052 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [MS][coreCloseExecutor-54-thread-1]: now merge
[junit4] 2> 155053 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [MS][coreCloseExecutor-54-thread-1]: index: _7(6.6.1):C383/382:delGen=1 _8(6.6.1):C10/10:delGen=1
[junit4] 2> 155053 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [MS][coreCloseExecutor-54-thread-1]: no more merges pending; now return
[junit4] 2> 155053 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-54-thread-1]: waitForMerges
[junit4] 2> 155053 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-54-thread-1]: waitForMerges done
[junit4] 2> 155053 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-54-thread-1]: commit: start
[junit4] 2> 155053 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-54-thread-1]: commit: enter lock
[junit4] 2> 155053 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-54-thread-1]: commit: now prepare
[junit4] 2> 155053 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-54-thread-1]: prepareCommit: flush
[junit4] 2> 155053 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-54-thread-1]: index before flush _7(6.6.1):C383/382:delGen=1 _8(6.6.1):C10/10:delGen=1
[junit4] 2> 155053 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [DW][coreCloseExecutor-54-thread-1]: startFullFlush
[junit4] 2> 155053 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-54-thread-1]: apply all deletes during flush
[junit4] 2> 155053 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-54-thread-1]: now apply all deletes for all segments maxDoc=393
[junit4] 2> 155053 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [BD][coreCloseExecutor-54-thread-1]: applyDeletes: open segment readers took 0 msec
[junit4] 2> 155053 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [BD][coreCloseExecutor-54-thread-1]: applyDeletes: no segments; skipping
[junit4] 2> 155053 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [BD][coreCloseExecutor-54-thread-1]: prune sis=segments_7: _7(6.6.1):C383/382:delGen=1 _8(6.6.1):C10/10:delGen=1 minGen=34 packetCount=0
[junit4] 2> 155053 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [DW][coreCloseExecutor-54-thread-1]: coreCloseExecutor-54-thread-1 finishFullFlush success=true
[junit4] 2> 155053 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-54-thread-1]: startCommit(): start
[junit4] 2> 155053 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-54-thread-1]: skip startCommit(): no changes pending
[junit4] 2> 155053 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreCloseExecutor-54-thread-1]: delete []
[junit4] 2> 155053 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-54-thread-1]: commit: pendingCommit == null; skip
[junit4] 2> 155053 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-54-thread-1]: commit: took 0.5 msec
[junit4] 2> 155053 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-54-thread-1]: commit: done
[junit4] 2> 155053 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-54-thread-1]: rollback
[junit4] 2> 155053 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-54-thread-1]: all running merges have aborted
[junit4] 2> 155053 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-54-thread-1]: rollback: done finish merges
[junit4] 2> 155053 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [DW][coreCloseExecutor-54-thread-1]: abort
[junit4] 2> 155053 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [DW][coreCloseExecutor-54-thread-1]: done abort success=true
[junit4] 2> 155053 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IW][coreCloseExecutor-54-thread-1]: rollback: infos=_7(6.6.1):C383/382:delGen=1 _8(6.6.1):C10/10:delGen=1
[junit4] 2> 155054 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreCloseExecutor-54-thread-1]: now checkpoint "_7(6.6.1):C383/382:delGen=1 _8(6.6.1):C10/10:delGen=1" [2 segments ; isCommit = false]
[junit4] 2> 155054 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreCloseExecutor-54-thread-1]: delete []
[junit4] 2> 155054 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreCloseExecutor-54-thread-1]: 0 msec to checkpoint
[junit4] 2> 155054 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreCloseExecutor-54-thread-1]: delete []
[junit4] 2> 155054 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.u.LoggingInfoStream [IFD][coreCloseExecutor-54-thread-1]: delete []
[junit4] 2> 155689 INFO (coreCloseExecutor-54-thread-1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.collection1
[junit4] 2> 156059 INFO (SUITE-TestSolrJErrorHandling-seed#[D635EEA6D10DC868]-worker) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@51252054{/solr,null,UNAVAILABLE}
[junit4] 2> 156060 INFO (SUITE-TestSolrJErrorHandling-seed#[D635EEA6D10DC868]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.TestSolrJErrorHandling_D635EEA6D10DC868-001
[junit4] 2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarity(queryNorm=false,coord=yes): {}, locale=es-PR, timezone=Pacific/Guadalcanal
[junit4] 2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=219976816,total=392167424
[junit4] 2> NOTE: All tests run in this JVM: [SolrDocumentTest, UuidEvaluatorTest, HttpSolrClientBuilderTest, FloorEvaluatorTest, SelectWithEvaluatorsTest, SolrQueryTest, CoalesceEvaluatorTest, StreamExpressionToExplanationTest, DocumentAnalysisResponseTest, CommonAdminParamsTest, SolrZkClientTest, ConcurrentUpdateSolrClientBuilderTest, TestSolrJErrorHandling]
[junit4] Completed [46/130 (1!)] on J1 in 104.10s, 4 tests, 1 failure <<< FAILURES!
[...truncated 42918 lines...]
[JENKINS] Lucene-Solr-NightlyTests-6.6 - Build # 25 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.6/25/
7 tests failed.
FAILED: org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([DF9907CEF3667FCC]:0)
FAILED: org.apache.solr.cloud.ShardSplitTest.testSplitAfterFailedSplit
Error Message:
expected:<1> but was:<2>
Stack Trace:
java.lang.AssertionError: expected:<1> but was:<2>
at __randomizedtesting.SeedInfo.seed([87EA478ACF01E0A7:7EA7D425F374AD2D]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.ShardSplitTest.testSplitAfterFailedSplit(ShardSplitTest.java:280)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest
Error Message:
ObjectTracker found 1 object(s) that were not released!!! [HdfsTransactionLog] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.update.HdfsTransactionLog at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.update.HdfsTransactionLog.<init>(HdfsTransactionLog.java:130) at org.apache.solr.update.HdfsTransactionLog.<init>(HdfsTransactionLog.java:74) at org.apache.solr.update.HdfsUpdateLog.ensureLog(HdfsUpdateLog.java:316) at org.apache.solr.update.UpdateLog.delete(UpdateLog.java:594) at org.apache.solr.update.DirectUpdateHandler2.delete(DirectUpdateHandler2.java:410) at org.apache.solr.update.processor.RunUpdateProcessor.processDelete(RunUpdateProcessorFactory.java:75) at org.apache.solr.update.processor.UpdateRequestProcessor.processDelete(UpdateRequestProcessor.java:59) at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalDelete(DistributedUpdateProcessor.java:984) at org.apache.solr.update.processor.DistributedUpdateProcessor.versionDelete(DistributedUpdateProcessor.java:1843) at org.apache.solr.update.processor.DistributedUpdateProcessor.processDelete(DistributedUpdateProcessor.java:1402) at org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.processDelete(LogUpdateProcessorFactory.java:124) at org.apache.solr.handler.loader.JavabinLoader.delete(JavabinLoader.java:152) at org.apache.solr.handler.loader.JavabinLoader.parseAndLoadDocs(JavabinLoader.java:113) at org.apache.solr.handler.loader.JavabinLoader.load(JavabinLoader.java:55) at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:97) at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:68) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173) at org.apache.solr.core.SolrCore.execute(SolrCore.java:2477) at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:723) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:529) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:361) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:305) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:136) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.eclipse.jetty.server.Server.handle(Server.java:534) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
java.lang.AssertionError: ObjectTracker found 1 object(s) that were not released!!! [HdfsTransactionLog]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.update.HdfsTransactionLog
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.update.HdfsTransactionLog.<init>(HdfsTransactionLog.java:130)
at org.apache.solr.update.HdfsTransactionLog.<init>(HdfsTransactionLog.java:74)
at org.apache.solr.update.HdfsUpdateLog.ensureLog(HdfsUpdateLog.java:316)
at org.apache.solr.update.UpdateLog.delete(UpdateLog.java:594)
at org.apache.solr.update.DirectUpdateHandler2.delete(DirectUpdateHandler2.java:410)
at org.apache.solr.update.processor.RunUpdateProcessor.processDelete(RunUpdateProcessorFactory.java:75)
at org.apache.solr.update.processor.UpdateRequestProcessor.processDelete(UpdateRequestProcessor.java:59)
at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalDelete(DistributedUpdateProcessor.java:984)
at org.apache.solr.update.processor.DistributedUpdateProcessor.versionDelete(DistributedUpdateProcessor.java:1843)
at org.apache.solr.update.processor.DistributedUpdateProcessor.processDelete(DistributedUpdateProcessor.java:1402)
at org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.processDelete(LogUpdateProcessorFactory.java:124)
at org.apache.solr.handler.loader.JavabinLoader.delete(JavabinLoader.java:152)
at org.apache.solr.handler.loader.JavabinLoader.parseAndLoadDocs(JavabinLoader.java:113)
at org.apache.solr.handler.loader.JavabinLoader.load(JavabinLoader.java:55)
at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:97)
at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:68)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2477)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:723)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:529)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:361)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:305)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:136)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:395)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:534)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([87EA478ACF01E0A7]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:302)
at sun.reflect.GeneratedMethodAccessor48.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.metrics.reporters.SolrJmxReporterCloudTest
Error Message:
Clean up static fields (in @AfterClass?), your test seems to hang on to approximately 19,346,048 bytes (threshold is 10,485,760). Field reference sizes (counted individually): - 19,345,544 bytes, private static javax.management.MBeanServer org.apache.solr.metrics.reporters.SolrJmxReporterCloudTest.mBeanServer - 192 bytes, public static org.junit.rules.TestRule org.apache.solr.SolrTestCaseJ4.solrClassRules - 128 bytes, private static java.lang.String org.apache.solr.SolrTestCaseJ4.factoryProp - 112 bytes, private static java.lang.String org.apache.solr.metrics.reporters.SolrJmxReporterCloudTest.COLLECTION - 64 bytes, private static java.lang.String org.apache.solr.SolrTestCaseJ4.coreName - 48 bytes, private static java.util.Map org.apache.solr.SolrTestCaseJ4.savedClassLogLevels
Stack Trace:
junit.framework.AssertionFailedError: Clean up static fields (in @AfterClass?), your test seems to hang on to approximately 19,346,048 bytes (threshold is 10,485,760). Field reference sizes (counted individually):
- 19,345,544 bytes, private static javax.management.MBeanServer org.apache.solr.metrics.reporters.SolrJmxReporterCloudTest.mBeanServer
- 192 bytes, public static org.junit.rules.TestRule org.apache.solr.SolrTestCaseJ4.solrClassRules
- 128 bytes, private static java.lang.String org.apache.solr.SolrTestCaseJ4.factoryProp
- 112 bytes, private static java.lang.String org.apache.solr.metrics.reporters.SolrJmxReporterCloudTest.COLLECTION
- 64 bytes, private static java.lang.String org.apache.solr.SolrTestCaseJ4.coreName
- 48 bytes, private static java.util.Map org.apache.solr.SolrTestCaseJ4.savedClassLogLevels
at __randomizedtesting.SeedInfo.seed([87EA478ACF01E0A7]:0)
at com.carrotsearch.randomizedtesting.rules.StaticFieldsInvariantRule$1.afterAlways(StaticFieldsInvariantRule.java:170)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.lucene.spatial3d.TestGeo3DPoint
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([DF9907CEF3667FCC]:0)
FAILED: org.apache.solr.cloud.hdfs.HdfsUnloadDistributedZkTest.test
Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:46302/_/oc
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:46302/_/oc
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:623)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at org.apache.solr.cloud.UnloadDistributedZkTest.testUnloadShardAndCollection(UnloadDistributedZkTest.java:146)
at org.apache.solr.cloud.UnloadDistributedZkTest.test(UnloadDistributedZkTest.java:71)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:515)
... 46 more
FAILED: org.apache.solr.common.cloud.TestCollectionStateWatchers.testWaitForStateWatcherIsRetainedOnPredicateFailure
Error Message:
Did not see a fully active cluster after 30 seconds
Stack Trace:
java.lang.AssertionError: Did not see a fully active cluster after 30 seconds
at __randomizedtesting.SeedInfo.seed([83CDCC1501C33EB1:BFB6E46D96CD6A3]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.common.cloud.TestCollectionStateWatchers.testWaitForStateWatcherIsRetainedOnPredicateFailure(TestCollectionStateWatchers.java:250)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 10416 lines...]
[junit4] Suite: org.apache.lucene.spatial3d.TestGeo3DPoint
[junit4] 2> aug 21, 2017 5:40:12 PM com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
[junit4] 2> WARNING: Suite execution timed out: org.apache.lucene.spatial3d.TestGeo3DPoint
[junit4] 2> 1) Thread[id=1, name=main, state=WAITING, group=main]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Thread.join(Thread.java:1252)
[junit4] 2> at java.lang.Thread.join(Thread.java:1326)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:608)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:457)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:244)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:355)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:13)
[junit4] 2> 2) Thread[id=13, name=SUITE-TestGeo3DPoint-seed#[DF9907CEF3667FCC], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at java.lang.Thread.getStackTrace(Thread.java:1559)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:696)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:693)
[junit4] 2> at java.security.AccessController.doPrivileged(Native Method)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getStackTrace(ThreadLeakControl.java:693)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getThreadsWithTraces(ThreadLeakControl.java:709)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:689)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:65)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:415)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:678)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:140)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:598)
[junit4] 2> 3) Thread[id=14, name=TEST-TestGeo3DPoint.testRandomBig-seed#[DF9907CEF3667FCC], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at org.apache.lucene.spatial3d.geom.Plane.addPoint(Plane.java:1953)
[junit4] 2> at org.apache.lucene.spatial3d.geom.Plane.recordBounds(Plane.java:1418)
[junit4] 2> at org.apache.lucene.spatial3d.geom.XYZBounds.addPlane(XYZBounds.java:161)
[junit4] 2> at org.apache.lucene.spatial3d.geom.XYZBounds.addHorizontalPlane(XYZBounds.java:178)
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoRectangle.getBounds(GeoRectangle.java:202)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.verify(TestGeo3DPoint.java:845)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.doTestRandom(TestGeo3DPoint.java:548)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig(TestGeo3DPoint.java:480)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 4) Thread[id=9, name=JUnit4-serializer-daemon, state=TIMED_WAITING, group=main]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:50)
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestGeo3DPoint -Dtests.method=testRandomBig -Dtests.seed=DF9907CEF3667FCC -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/test-data/enwiki.random.lines.txt -Dtests.locale=sk-SK -Dtests.timezone=Arctic/Longyearbyen -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 6699s J1 | TestGeo3DPoint.testRandomBig <<<
[junit4] > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
[junit4] > at __randomizedtesting.SeedInfo.seed([DF9907CEF3667FCC]:0)
[junit4] 2> aug 21, 2017 5:40:12 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> aug 21, 2017 5:40:32 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.lucene.spatial3d.TestGeo3DPoint:
[junit4] 2> 1) Thread[id=14, name=TEST-TestGeo3DPoint.testRandomBig-seed#[DF9907CEF3667FCC], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at org.apache.lucene.spatial3d.geom.Plane.findIntersectionBounds(Plane.java:1097)
[junit4] 2> at org.apache.lucene.spatial3d.geom.Plane.recordBounds(Plane.java:1214)
[junit4] 2> at org.apache.lucene.spatial3d.geom.XYZBounds.addIntersection(XYZBounds.java:262)
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoConvexPolygon.getBounds(GeoConvexPolygon.java:425)
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoCompositeMembershipShape.getBounds(GeoCompositeMembershipShape.java:75)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.verify(TestGeo3DPoint.java:845)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.doTestRandom(TestGeo3DPoint.java:548)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig(TestGeo3DPoint.java:480)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> aug 21, 2017 5:40:32 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=14, name=TEST-TestGeo3DPoint.testRandomBig-seed#[DF9907CEF3667FCC], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/lucene/build/spatial3d/test/J1/temp/lucene.spatial3d.TestGeo3DPoint_DF9907CEF3667FCC-001
[junit4] 2> Aug 21, 2017 3:40:32 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: All leaked threads terminated.
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): {id=PostingsFormat(name=LuceneVarGapDocFreqInterval)}, docValues:{id=DocValuesFormat(name=Lucene54), point=DocValuesFormat(name=Direct)}, maxPointsInLeafNode=27, maxMBSortInHeap=6.212016999483232, sim=RandomSimilarity(queryNorm=false,coord=crazy): {}, locale=sk-SK, timezone=Arctic/Longyearbyen
[junit4] 2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=200081528,total=448266240
[junit4] 2> NOTE: All tests run in this JVM: [GeoPointTest, TestGeo3DPoint]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestGeo3DPoint -Dtests.seed=DF9907CEF3667FCC -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/test-data/enwiki.random.lines.txt -Dtests.locale=sk-SK -Dtests.timezone=Arctic/Longyearbyen -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J1 | TestGeo3DPoint (suite) <<<
[junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4] > at __randomizedtesting.SeedInfo.seed([DF9907CEF3667FCC]:0)
[junit4] Completed [11/11 (1!)] on J1 in 7220.57s, 9 tests, 2 errors <<< FAILURES!
[...truncated 1809 lines...]
[junit4] Suite: org.apache.solr.cloud.ShardSplitTest
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_87EA478ACF01E0A7-001/init-core-data-001
[junit4] 2> 112503 WARN (SUITE-ShardSplitTest-seed#[87EA478ACF01E0A7]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=2 numCloses=2
[junit4] 2> 112533 INFO (SUITE-ShardSplitTest-seed#[87EA478ACF01E0A7]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields
[junit4] 2> 112535 INFO (SUITE-ShardSplitTest-seed#[87EA478ACF01E0A7]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
[junit4] 2> 112535 INFO (SUITE-ShardSplitTest-seed#[87EA478ACF01E0A7]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /s_/a
[junit4] 2> 112582 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 112585 INFO (Thread-67) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 112650 INFO (Thread-67) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 112797 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.c.ZkTestServer start zk server on port:35106
[junit4] 2> 112829 ERROR (Thread-67) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 113026 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 113106 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 113122 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 113123 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 113124 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 113141 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 113154 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 113155 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 113156 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 113157 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 113187 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 113250 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e057cfc070001, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 114296 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_87EA478ACF01E0A7-001/control-001/cores/collection1
[junit4] 2> 114298 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 114380 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4458b0ca{/s_/a,null,AVAILABLE}
[junit4] 2> 114381 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@30d59b90{HTTP/1.1,[http/1.1]}{127.0.0.1:43374}
[junit4] 2> 114381 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.e.j.s.Server Started @145331ms
[junit4] 2> 114381 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_87EA478ACF01E0A7-001/tempDir-001/control/data, hostContext=/s_/a, hostPort=43374, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_87EA478ACF01E0A7-001/control-001/cores}
[junit4] 2> 114381 ERROR (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 114381 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.6.1
[junit4] 2> 114381 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 114381 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 114381 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-21T15:50:32.296Z
[junit4] 2> 114510 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 114510 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_87EA478ACF01E0A7-001/control-001/solr.xml
[junit4] 2> 114672 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 114673 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35106/solr
[junit4] 2> 115004 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [n:127.0.0.1:43374_s_%2Fa ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 115051 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [n:127.0.0.1:43374_s_%2Fa ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:43374_s_%2Fa
[junit4] 2> 115052 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [n:127.0.0.1:43374_s_%2Fa ] o.a.s.c.Overseer Overseer (id=98522276211130372-127.0.0.1:43374_s_%2Fa-n_0000000000) starting
[junit4] 2> 115265 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [n:127.0.0.1:43374_s_%2Fa ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43374_s_%2Fa
[junit4] 2> 115297 INFO (zkCallback-48-thread-1-processing-n:127.0.0.1:43374_s_%2Fa) [n:127.0.0.1:43374_s_%2Fa ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 116054 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [n:127.0.0.1:43374_s_%2Fa ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_87EA478ACF01E0A7-001/control-001/cores
[junit4] 2> 116055 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [n:127.0.0.1:43374_s_%2Fa ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 116166 INFO (coreLoadExecutor-121-thread-1-processing-n:127.0.0.1:43374_s_%2Fa) [n:127.0.0.1:43374_s_%2Fa c:control_collection x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 116169 INFO (OverseerStateUpdate-98522276211130372-127.0.0.1:43374_s_%2Fa-n_0000000000) [n:127.0.0.1:43374_s_%2Fa ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 116411 INFO (coreLoadExecutor-121-thread-1-processing-n:127.0.0.1:43374_s_%2Fa) [n:127.0.0.1:43374_s_%2Fa c:control_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.1
[junit4] 2> 116564 INFO (coreLoadExecutor-121-thread-1-processing-n:127.0.0.1:43374_s_%2Fa) [n:127.0.0.1:43374_s_%2Fa c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 117677 INFO (coreLoadExecutor-121-thread-1-processing-n:127.0.0.1:43374_s_%2Fa) [n:127.0.0.1:43374_s_%2Fa c:control_collection x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 118183 INFO (coreLoadExecutor-121-thread-1-processing-n:127.0.0.1:43374_s_%2Fa) [n:127.0.0.1:43374_s_%2Fa c:control_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection, trusted=true
[junit4] 2> 118216 INFO (coreLoadExecutor-121-thread-1-processing-n:127.0.0.1:43374_s_%2Fa) [n:127.0.0.1:43374_s_%2Fa c:control_collection x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 118216 INFO (coreLoadExecutor-121-thread-1-processing-n:127.0.0.1:43374_s_%2Fa) [n:127.0.0.1:43374_s_%2Fa c:control_collection x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_87EA478ACF01E0A7-001/control-001/cores/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_87EA478ACF01E0A7-001/control-001/cores/collection1/data/]
[junit4] 2> 118216 INFO (coreLoadExecutor-121-thread-1-processing-n:127.0.0.1:43374_s_%2Fa) [n:127.0.0.1:43374_s_%2Fa c:control_collection x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5dfbad7d
[junit4] 2> 118231 INFO (coreLoadExecutor-121-thread-1-processing-n:127.0.0.1:43374_s_%2Fa) [n:127.0.0.1:43374_s_%2Fa c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@5936911a
[junit4] 2> 118262 WARN (coreLoadExecutor-121-thread-1-processing-n:127.0.0.1:43374_s_%2Fa) [n:127.0.0.1:43374_s_%2Fa c:control_collection x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 118458 INFO (coreLoadExecutor-121-thread-1-processing-n:127.0.0.1:43374_s_%2Fa) [n:127.0.0.1:43374_s_%2Fa c:control_collection x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 118458 INFO (coreLoadExecutor-121-thread-1-processing-n:127.0.0.1:43374_s_%2Fa) [n:127.0.0.1:43374_s_%2Fa c:control_collection x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 118459 INFO (coreLoadExecutor-121-thread-1-processing-n:127.0.0.1:43374_s_%2Fa) [n:127.0.0.1:43374_s_%2Fa c:control_collection x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 118459 INFO (coreLoadExecutor-121-thread-1-processing-n:127.0.0.1:43374_s_%2Fa) [n:127.0.0.1:43374_s_%2Fa c:control_collection x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 118460 INFO (coreLoadExecutor-121-thread-1-processing-n:127.0.0.1:43374_s_%2Fa) [n:127.0.0.1:43374_s_%2Fa c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=23, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8603680928257668]
[junit4] 2> 118461 INFO (coreLoadExecutor-121-thread-1-processing-n:127.0.0.1:43374_s_%2Fa) [n:127.0.0.1:43374_s_%2Fa c:control_collection x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@590be8d8[collection1] main]
[junit4] 2> 118474 INFO (coreLoadExecutor-121-thread-1-processing-n:127.0.0.1:43374_s_%2Fa) [n:127.0.0.1:43374_s_%2Fa c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 118475 INFO (coreLoadExecutor-121-thread-1-processing-n:127.0.0.1:43374_s_%2Fa) [n:127.0.0.1:43374_s_%2Fa c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 118475 INFO (coreLoadExecutor-121-thread-1-processing-n:127.0.0.1:43374_s_%2Fa) [n:127.0.0.1:43374_s_%2Fa c:control_collection x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 118476 INFO (coreLoadExecutor-121-thread-1-processing-n:127.0.0.1:43374_s_%2Fa) [n:127.0.0.1:43374_s_%2Fa c:control_collection x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1576356425384329216
[junit4] 2> 118523 INFO (coreZkRegister-114-thread-1-processing-n:127.0.0.1:43374_s_%2Fa x:collection1 c:control_collection) [n:127.0.0.1:43374_s_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 118523 INFO (coreZkRegister-114-thread-1-processing-n:127.0.0.1:43374_s_%2Fa x:collection1 c:control_collection) [n:127.0.0.1:43374_s_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 118523 INFO (coreZkRegister-114-thread-1-processing-n:127.0.0.1:43374_s_%2Fa x:collection1 c:control_collection) [n:127.0.0.1:43374_s_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:43374/s_/a/collection1/
[junit4] 2> 118523 INFO (coreZkRegister-114-thread-1-processing-n:127.0.0.1:43374_s_%2Fa x:collection1 c:control_collection) [n:127.0.0.1:43374_s_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 118523 INFO (coreZkRegister-114-thread-1-processing-n:127.0.0.1:43374_s_%2Fa x:collection1 c:control_collection) [n:127.0.0.1:43374_s_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:43374/s_/a/collection1/ has no replicas
[junit4] 2> 118523 INFO (coreZkRegister-114-thread-1-processing-n:127.0.0.1:43374_s_%2Fa x:collection1 c:control_collection) [n:127.0.0.1:43374_s_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 118526 INFO (coreZkRegister-114-thread-1-processing-n:127.0.0.1:43374_s_%2Fa x:collection1 c:control_collection) [n:127.0.0.1:43374_s_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:43374/s_/a/collection1/ shard1
[junit4] 2> 118592 INFO (searcherExecutor-122-thread-1-processing-n:127.0.0.1:43374_s_%2Fa x:collection1 c:control_collection) [n:127.0.0.1:43374_s_%2Fa c:control_collection x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@590be8d8[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 118811 INFO (coreZkRegister-114-thread-1-processing-n:127.0.0.1:43374_s_%2Fa x:collection1 c:control_collection) [n:127.0.0.1:43374_s_%2Fa c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 119112 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 119177 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:35106/solr ready
[junit4] 2> 119177 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 119736 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_87EA478ACF01E0A7-001/shard-1-001/cores/collection1
[junit4] 2> 119736 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_87EA478ACF01E0A7-001/shard-1-001
[junit4] 2> 119756 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 119757 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2a2199b5{/s_/a,null,AVAILABLE}
[junit4] 2> 119757 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@118516ca{HTTP/1.1,[http/1.1]}{127.0.0.1:39759}
[junit4] 2> 119757 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.e.j.s.Server Started @150708ms
[junit4] 2> 119757 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_87EA478ACF01E0A7-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/s_/a, hostPort=39759, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_87EA478ACF01E0A7-001/shard-1-001/cores}
[junit4] 2> 119799 ERROR (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 119819 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.6.1
[junit4] 2> 119819 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 119819 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 119819 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-21T15:50:37.734Z
[junit4] 2> 119941 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 119941 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_87EA478ACF01E0A7-001/shard-1-001/solr.xml
[junit4] 2> 120152 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 120154 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35106/solr
[junit4] 2> 120303 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [n:127.0.0.1:39759_s_%2Fa ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 120306 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [n:127.0.0.1:39759_s_%2Fa ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 120356 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [n:127.0.0.1:39759_s_%2Fa ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39759_s_%2Fa
[junit4] 2> 120437 INFO (zkCallback-57-thread-1-processing-n:127.0.0.1:39759_s_%2Fa) [n:127.0.0.1:39759_s_%2Fa ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 120461 INFO (zkCallback-48-thread-3-processing-n:127.0.0.1:43374_s_%2Fa) [n:127.0.0.1:43374_s_%2Fa ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 120526 INFO (zkCallback-52-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 122256 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [n:127.0.0.1:39759_s_%2Fa ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_87EA478ACF01E0A7-001/shard-1-001/cores
[junit4] 2> 122256 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [n:127.0.0.1:39759_s_%2Fa ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 122403 INFO (coreLoadExecutor-132-thread-1-processing-n:127.0.0.1:39759_s_%2Fa) [n:127.0.0.1:39759_s_%2Fa c:collection1 x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 122447 INFO (OverseerStateUpdate-98522276211130372-127.0.0.1:43374_s_%2Fa-n_0000000000) [n:127.0.0.1:43374_s_%2Fa ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 123441 INFO (coreLoadExecutor-132-thread-1-processing-n:127.0.0.1:39759_s_%2Fa) [n:127.0.0.1:39759_s_%2Fa c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.1
[junit4] 2> 123520 INFO (coreLoadExecutor-132-thread-1-processing-n:127.0.0.1:39759_s_%2Fa) [n:127.0.0.1:39759_s_%2Fa c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 124304 INFO (coreLoadExecutor-132-thread-1-processing-n:127.0.0.1:39759_s_%2Fa) [n:127.0.0.1:39759_s_%2Fa c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 124436 INFO (coreLoadExecutor-132-thread-1-processing-n:127.0.0.1:39759_s_%2Fa) [n:127.0.0.1:39759_s_%2Fa c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1, trusted=true
[junit4] 2> 124437 INFO (coreLoadExecutor-132-thread-1-processing-n:127.0.0.1:39759_s_%2Fa) [n:127.0.0.1:39759_s_%2Fa c:collection1 x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 124437 INFO (coreLoadExecutor-132-thread-1-processing-n:127.0.0.1:39759_s_%2Fa) [n:127.0.0.1:39759_s_%2Fa c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_87EA478ACF01E0A7-001/shard-1-001/cores/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_87EA478ACF01E0A7-001/shard-1-001/cores/collection1/data/]
[junit4] 2> 124437 INFO (coreLoadExecutor-132-thread-1-processing-n:127.0.0.1:39759_s_%2Fa) [n:127.0.0.1:39759_s_%2Fa c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5dfbad7d
[junit4] 2> 124480 INFO (coreLoadExecutor-132-thread-1-processing-n:127.0.0.1:39759_s_%2Fa) [n:127.0.0.1:39759_s_%2Fa c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@2dcdc4a3
[junit4] 2> 124613 WARN (coreLoadExecutor-132-thread-1-processing-n:127.0.0.1:39759_s_%2Fa) [n:127.0.0.1:39759_s_%2Fa c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 124970 INFO (coreLoadExecutor-132-thread-1-processing-n:127.0.0.1:39759_s_%2Fa) [n:127.0.0.1:39759_s_%2Fa c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 124970 INFO (coreLoadExecutor-132-thread-1-processing-n:127.0.0.1:39759_s_%2Fa) [n:127.0.0.1:39759_s_%2Fa c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 124971 INFO (coreLoadExecutor-132-thread-1-processing-n:127.0.0.1:39759_s_%2Fa) [n:127.0.0.1:39759_s_%2Fa c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 124971 INFO (coreLoadExecutor-132-thread-1-processing-n:127.0.0.1:39759_s_%2Fa) [n:127.0.0.1:39759_s_%2Fa c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 125019 INFO (coreLoadExecutor-132-thread-1-processing-n:127.0.0.1:39759_s_%2Fa) [n:127.0.0.1:39759_s_%2Fa c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=23, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8603680928257668]
[junit4] 2> 125021 INFO (coreLoadExecutor-132-thread-1-processing-n:127.0.0.1:39759_s_%2Fa) [n:127.0.0.1:39759_s_%2Fa c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@53f6512b[collection1] main]
[junit4] 2> 125021 INFO (coreLoadExecutor-132-thread-1-processing-n:127.0.0.1:39759_s_%2Fa) [n:127.0.0.1:39759_s_%2Fa c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 125022 INFO (coreLoadExecutor-132-thread-1-processing-n:127.0.0.1:39759_s_%2Fa) [n:127.0.0.1:39759_s_%2Fa c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 125022 INFO (coreLoadExecutor-132-thread-1-processing-n:127.0.0.1:39759_s_%2Fa) [n:127.0.0.1:39759_s_%2Fa c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 125084 INFO (searcherExecutor-133-thread-1-processing-n:127.0.0.1:39759_s_%2Fa x:collection1 c:collection1) [n:127.0.0.1:39759_s_%2Fa c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@53f6512b[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 125085 INFO (coreLoadExecutor-132-thread-1-processing-n:127.0.0.1:39759_s_%2Fa) [n:127.0.0.1:39759_s_%2Fa c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1576356432314368000
[junit4] 2> 125184 INFO (coreZkRegister-127-thread-1-processing-n:127.0.0.1:39759_s_%2Fa x:collection1 c:collection1) [n:127.0.0.1:39759_s_%2Fa c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 125184 INFO (coreZkRegister-127-thread-1-processing-n:127.0.0.1:39759_s_%2Fa x:collection1 c:collection1) [n:127.0.0.1:39759_s_%2Fa c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 125184 INFO (coreZkRegister-127-thread-1-processing-n:127.0.0.1:39759_s_%2Fa x:collection1 c:collection1) [n:127.0.0.1:39759_s_%2Fa c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:39759/s_/a/collection1/
[junit4] 2> 125184 INFO (coreZkRegister-127-thread-1-processing-n:127.0.0.1:39759_s_%2Fa x:collection1 c:collection1) [n:127.0.0.1:39759_s_%2Fa c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 125184 INFO (coreZkRegister-127-thread-1-processing-n:127.0.0.1:39759_s_%2Fa x:collection1 c:collection1) [n:127.0.0.1:39759_s_%2Fa c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:39759/s_/a/collection1/ has no replicas
[junit4] 2> 125184 INFO (coreZkRegister-127-thread-1-processing-n:127.0.0.1:39759_s_%2Fa x:collection1 c:collection1) [n:127.0.0.1:39759_s_%2Fa c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 125201 INFO (coreZkRegister-127-thread-1-processing-n:127.0.0.1:39759_s_%2Fa x:collection1 c:collection1) [n:127.0.0.1:39759_s_%2Fa c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:39759/s_/a/collection1/ shard2
[junit4] 2> 125409 INFO (coreZkRegister-127-thread-1-processing-n:127.0.0.1:39759_s_%2Fa x:collection1 c:collection1) [n:127.0.0.1:39759_s_%2Fa c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 125903 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_87EA478ACF01E0A7-001/shard-2-001/cores/collection1
[junit4] 2> 125904 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_87EA478ACF01E0A7-001/shard-2-001
[junit4] 2> 125917 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 125918 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2fda593{/s_/a,null,AVAILABLE}
[junit4] 2> 125919 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7e6fa6d0{HTTP/1.1,[http/1.1]}{127.0.0.1:45012}
[junit4] 2> 125919 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.e.j.s.Server Started @156869ms
[junit4] 2> 125919 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_87EA478ACF01E0A7-001/tempDir-001/jetty2, solrconfig=solrconfig.xml, hostContext=/s_/a, hostPort=45012, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_87EA478ACF01E0A7-001/shard-2-001/cores}
[junit4] 2> 125919 ERROR (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 125929 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.6.1
[junit4] 2> 125929 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 125929 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 125929 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-21T15:50:43.844Z
[junit4] 2> 125964 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 125964 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_87EA478ACF01E0A7-001/shard-2-001/solr.xml
[junit4] 2> 125998 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 125999 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35106/solr
[junit4] 2> 126196 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [n:127.0.0.1:45012_s_%2Fa ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 126210 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [n:127.0.0.1:45012_s_%2Fa ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 126212 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [n:127.0.0.1:45012_s_%2Fa ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45012_s_%2Fa
[junit4] 2> 126226 INFO (zkCallback-52-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 126311 INFO (zkCallback-57-thread-1-processing-n:127.0.0.1:39759_s_%2Fa) [n:127.0.0.1:39759_s_%2Fa ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 126374 INFO (zkCallback-63-thread-1-processing-n:127.0.0.1:45012_s_%2Fa) [n:127.0.0.1:45012_s_%2Fa ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 126374 INFO (zkCallback-48-thread-1-processing-n:127.0.0.1:43374_s_%2Fa) [n:127.0.0.1:43374_s_%2Fa ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 126770 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [n:127.0.0.1:45012_s_%2Fa ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_87EA478ACF01E0A7-001/shard-2-001/cores
[junit4] 2> 126770 INFO (TEST-ShardSplitTest.testSplitWithChaosMonkey-seed#[87EA478ACF01E0A7]) [n:127.0.0.1:45012_s_%2Fa ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 126872 INFO (coreLoadExecutor-143-thread-1-processing-n:127.0.0.1:45012_s_%2Fa) [n:127.0.0.1:45012_s_%2Fa c:collection1 x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 126913 INFO (OverseerStateUpdate-98522276211130372-127.0.0.1:43374_s_%2Fa-n_0000000000) [n:127.0.0.1:43374_s_%2Fa ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 128075 INFO (coreLoadExecutor-143-thread-1-processing-n:127.0.0.1:45012_s_%2Fa) [n:127.0.0.1:45012_s_%2Fa c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.1
[junit4] 2> 128133 INFO (coreLoadExecutor-143-thread-1-processing-n:127.0.0.1:45012_s_%2Fa) [n:127.0.0.1:45012_s_%2Fa c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 128557 INFO (coreLoadExecutor-143-thread-1-processing-n:127.0.0.1:45012_s_%2Fa) [n:127.0.0.1:45012_s_%2Fa c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 128711 INFO (coreLoadExecutor-143-thread-1-processing-n:127.0.0.1:45012_s_%2Fa) [n:127.0.0.1:45012_s_%2Fa c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1, trusted=true
[junit4] 2> 128711 INFO (coreLoadExecutor-143-thread-1-processing-n:127.0.0.1:45012_s_%2Fa) [n:127.0.0.1:45012_s_%2Fa c:collection1 x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 128711 INFO (coreLoadExecutor-143-thread-1-processing-n:127.0.0.1:45012_s_%2Fa) [n:127.0.0.1:45012_s_%2Fa c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_87EA478ACF01E0A7-001/shard-2-001/cores/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-core/test/J2/temp/solr.clo
[...truncated too long message...]
ayed s:shard1 r:core_node2 x:delayed_shard1_replica1] o.a.s.c.SolrCore [delayed_shard1_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@2698a481
[junit4] 2> 523664 INFO (coreCloseExecutor-392-thread-1) [n:127.0.0.1:41717_solr c:falsepredicate s:shard1 r:core_node5 x:falsepredicate_shard1_replica1] o.a.s.c.SolrCore [falsepredicate_shard1_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@72c5c5e3
[junit4] 2> 523664 INFO (coreCloseExecutor-392-thread-3) [n:127.0.0.1:41717_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica1] o.a.s.c.SolrCore [testcollection_shard1_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@1f768ff
[junit4] 2> 523724 INFO (coreCloseExecutor-390-thread-2) [n:127.0.0.1:34182_solr c:testcollection s:shard3 r:core_node1 x:testcollection_shard3_replica1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.testcollection.shard3.replica1
[junit4] 2> 523727 INFO (coreCloseExecutor-389-thread-1) [n:127.0.0.1:35480_solr c:currentstate s:shard1 r:core_node1 x:currentstate_shard1_replica1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.currentstate.shard1.replica1
[junit4] 2> 523731 INFO (coreCloseExecutor-392-thread-2) [n:127.0.0.1:41717_solr c:delayed s:shard1 r:core_node2 x:delayed_shard1_replica1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.delayed.shard1.replica1
[junit4] 2> 523734 INFO (zkCallback-1833-thread-2-processing-n:127.0.0.1:35480_solr) [n:127.0.0.1:35480_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/currentstate/state.json] for collection [currentstate] has occurred - updating... (live nodes size: [4])
[junit4] 2> 523735 INFO (zkCallback-1833-thread-1-processing-n:127.0.0.1:35480_solr) [n:127.0.0.1:35480_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/falsepredicate/state.json] for collection [falsepredicate] has occurred - updating... (live nodes size: [4])
[junit4] 2> 523736 INFO (jetty-closer-1792-thread-1) [ ] o.a.s.c.Overseer Overseer (id=98522989898498055-127.0.0.1:34182_solr-n_0000000002) closing
[junit4] 2> 523744 INFO (OverseerStateUpdate-98522989898498055-127.0.0.1:34182_solr-n_0000000002) [n:127.0.0.1:34182_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:34182_solr
[junit4] 2> 523768 INFO (coreCloseExecutor-392-thread-3) [n:127.0.0.1:41717_solr c:testcollection s:shard1 r:core_node5 x:testcollection_shard1_replica1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.testcollection.shard1.replica1
[junit4] 2> 523769 INFO (coreCloseExecutor-392-thread-1) [n:127.0.0.1:41717_solr c:falsepredicate s:shard1 r:core_node5 x:falsepredicate_shard1_replica1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.falsepredicate.shard1.replica1
[junit4] 2> 523770 INFO (coreCloseExecutor-391-thread-3) [n:127.0.0.1:41580_solr c:falsepredicate s:shard3 r:core_node6 x:falsepredicate_shard3_replica1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.falsepredicate.shard3.replica1
[junit4] 2> 523771 WARN (zkCallback-1833-thread-2-processing-n:127.0.0.1:35480_solr) [n:127.0.0.1:35480_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 523775 INFO (zkCallback-1847-thread-1-processing-n:127.0.0.1:41580_solr) [n:127.0.0.1:41580_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (2)
[junit4] 2> 523775 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e0623271c001d, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 523775 INFO (zkCallback-1801-thread-2-processing-n:127.0.0.1:34182_solr) [n:127.0.0.1:34182_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (2)
[junit4] 2> 523801 INFO (zkCallback-1847-thread-2-processing-n:127.0.0.1:41580_solr) [n:127.0.0.1:41580_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:41580_solr
[junit4] 2> 523804 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e0623271c0007, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 525236 WARN (zkCallback-1833-thread-1-processing-n:127.0.0.1:35480_solr) [n:127.0.0.1:35480_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /collections/falsepredicate/state.json]
[junit4] 2> 525236 INFO (jetty-closer-1792-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@d718ab3{/solr,null,UNAVAILABLE}
[junit4] 2> 525271 WARN (zkCallback-1841-thread-1-processing-n:127.0.0.1:41717_solr) [n:127.0.0.1:41717_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 525272 INFO (jetty-closer-1792-thread-3) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@27e1e359{/solr,null,UNAVAILABLE}
[junit4] 2> 525298 WARN (zkCallback-1801-thread-2-processing-n:127.0.0.1:34182_solr) [n:127.0.0.1:34182_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 525298 INFO (jetty-closer-1792-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@161222e{/solr,null,UNAVAILABLE}
[junit4] 2> 525301 WARN (zkCallback-1847-thread-1-processing-n:127.0.0.1:41580_solr) [n:127.0.0.1:41580_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 525302 INFO (jetty-closer-1792-thread-4) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@73f00ad9{/solr,null,UNAVAILABLE}
[junit4] 2> 525327 ERROR (SUITE-TestCollectionStateWatchers-seed#[83CDCC1501C33EB1]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 525327 INFO (SUITE-TestCollectionStateWatchers-seed#[83CDCC1501C33EB1]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:37774 37774
[junit4] 2> 525478 INFO (Thread-125) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:37774 37774
[junit4] 2> 525478 WARN (Thread-125) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 8 /solr/aliases.json
[junit4] 2> 7 /solr/security.json
[junit4] 2> 7 /solr/configs/config
[junit4] 2> 3 /solr/collections/stateformat1/state.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 12 /solr/collections/waitforstate/state.json
[junit4] 2> 8 /solr/clusterprops.json
[junit4] 2> 8 /solr/clusterstate.json
[junit4] 2> 8 /solr/collections/testcollection/state.json
[junit4] 2> 7 /solr/collections/falsepredicate/state.json
[junit4] 2> 3 /solr/collections/stateformat1/state.json
[junit4] 2> 2 /solr/overseer_elect/election/98522989898498059-127.0.0.1:37655_solr-n_0000000000
[junit4] 2> 2 /solr/collections/currentstate/state.json
[junit4] 2> 2 /solr/overseer_elect/election/98522989898498055-127.0.0.1:34182_solr-n_0000000002
[junit4] 2> 2 /solr/collections/delayed/state.json
[junit4] 2> 2 /solr/collections/tobedeleted/state.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 8 /solr/collections
[junit4] 2> 5 /solr/live_nodes
[junit4] 2> 2 /solr/overseer/queue
[junit4] 2> 2 /solr/overseer/collection-queue-work
[junit4] 2>
[junit4] 2> 525478 INFO (SUITE-TestCollectionStateWatchers-seed#[83CDCC1501C33EB1]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.6/checkout/solr/build/solr-solrj/test/J2/temp/solr.common.cloud.TestCollectionStateWatchers_83CDCC1501C33EB1-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): {}, docValues:{}, maxPointsInLeafNode=1491, maxMBSortInHeap=6.3627542858140815, sim=RandomSimilarity(queryNorm=false,coord=no): {}, locale=en-PH, timezone=Chile/Continental
[junit4] 2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=260486992,total=414711808
[junit4] 2> NOTE: All tests run in this JVM: [TestHash, SolrExceptionTest, StreamExpressionTest, JdbcTest, CubedRootEvaluatorTest, CloudSolrClientMultiConstructorTest, HttpSolrClientConPoolTest, JettyWebappTest, AndEvaluatorTest, LargeVolumeJettyTest, PowerEvaluatorTest, TestCollectionStateWatchers]
[junit4] Completed [119/130 (1!)] on J2 in 76.68s, 9 tests, 1 failure <<< FAILURES!
[...truncated 42678 lines...]