You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2017/08/11 12:05:43 UTC
[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 26 - Still
Unstable
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/26/
6 tests failed.
FAILED: org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomMedium
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([21D6FA9DD0AC0AF2]:0)
FAILED: junit.framework.TestSuite.org.apache.lucene.spatial3d.TestGeo3DPoint
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([21D6FA9DD0AC0AF2]:0)
FAILED: org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest.test
Error Message:
The Monkey ran for over 45 seconds and no jetties were stopped - this is worth investigating!
Stack Trace:
java.lang.AssertionError: The Monkey ran for over 45 seconds and no jetties were stopped - this is worth investigating!
at __randomizedtesting.SeedInfo.seed([D656E80D4531B37C:5E02D7D7EBCDDE84]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.ChaosMonkey.stopTheMonkey(ChaosMonkey.java:587)
at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.test(ChaosMonkeySafeLeaderTest.java:133)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest
Error Message:
2 threads leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest: 1) Thread[id=6169, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-HdfsChaosMonkeySafeLeaderTest] at java.lang.Thread.sleep(Native Method) at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135) 2) Thread[id=6168, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-HdfsChaosMonkeySafeLeaderTest] at java.lang.Thread.sleep(Native Method) at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest:
1) Thread[id=6169, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-HdfsChaosMonkeySafeLeaderTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135)
2) Thread[id=6168, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-HdfsChaosMonkeySafeLeaderTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135)
at __randomizedtesting.SeedInfo.seed([D656E80D4531B37C]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=6169, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-HdfsChaosMonkeySafeLeaderTest] at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135) 2) Thread[id=6168, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-HdfsChaosMonkeySafeLeaderTest] at java.security.AccessController.doPrivileged(Native Method) at com.carrotsearch.ant.tasks.junit4.events.Serializer.flushQueue(Serializer.java:114) at com.carrotsearch.ant.tasks.junit4.events.Serializer.serialize(Serializer.java:99) at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain$3$2.write(SlaveMain.java:472) at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) at java.io.PrintStream.flush(PrintStream.java:338) at java.io.FilterOutputStream.flush(FilterOutputStream.java:140) at java.io.PrintStream.write(PrintStream.java:482) at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221) at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291) at sun.nio.cs.StreamEncoder.flushBuffer(StreamEncoder.java:104) at java.io.OutputStreamWriter.flushBuffer(OutputStreamWriter.java:185) at java.io.PrintStream.write(PrintStream.java:527) at java.io.PrintStream.print(PrintStream.java:669) at java.io.PrintStream.println(PrintStream.java:823) at java.lang.Throwable$WrappedPrintStream.println(Throwable.java:748) at java.lang.Throwable.printStackTrace(Throwable.java:658) at java.lang.Throwable.printStackTrace(Throwable.java:643) at java.lang.Throwable.printStackTrace(Throwable.java:634) at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:121)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=6169, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-HdfsChaosMonkeySafeLeaderTest]
at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135)
2) Thread[id=6168, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-HdfsChaosMonkeySafeLeaderTest]
at java.security.AccessController.doPrivileged(Native Method)
at com.carrotsearch.ant.tasks.junit4.events.Serializer.flushQueue(Serializer.java:114)
at com.carrotsearch.ant.tasks.junit4.events.Serializer.serialize(Serializer.java:99)
at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain$3$2.write(SlaveMain.java:472)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
at java.io.PrintStream.flush(PrintStream.java:338)
at java.io.FilterOutputStream.flush(FilterOutputStream.java:140)
at java.io.PrintStream.write(PrintStream.java:482)
at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
at sun.nio.cs.StreamEncoder.flushBuffer(StreamEncoder.java:104)
at java.io.OutputStreamWriter.flushBuffer(OutputStreamWriter.java:185)
at java.io.PrintStream.write(PrintStream.java:527)
at java.io.PrintStream.print(PrintStream.java:669)
at java.io.PrintStream.println(PrintStream.java:823)
at java.lang.Throwable$WrappedPrintStream.println(Throwable.java:748)
at java.lang.Throwable.printStackTrace(Throwable.java:658)
at java.lang.Throwable.printStackTrace(Throwable.java:643)
at java.lang.Throwable.printStackTrace(Throwable.java:634)
at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:121)
at __randomizedtesting.SeedInfo.seed([D656E80D4531B37C]:0)
FAILED: org.apache.solr.client.solrj.impl.CloudSolrClientTest.testRouting
Error Message:
Could not load collection from ZK: collection1
Stack Trace:
org.apache.solr.common.SolrException: Could not load collection from ZK: collection1
at __randomizedtesting.SeedInfo.seed([67712A0B1DEDE8:C2D04D42085D1D90]:0)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1115)
at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:648)
at org.apache.solr.client.solrj.impl.CloudSolrClient.getDocCollection(CloudSolrClient.java:1227)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:827)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:793)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
at org.apache.solr.client.solrj.impl.CloudSolrClientTest.testRouting(CloudSolrClientTest.java:379)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:110)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/state.json
at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1212)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:357)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:354)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:354)
at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1127)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1113)
... 47 more
Build Log:
[...truncated 10015 lines...]
[junit4] Suite: org.apache.lucene.spatial3d.TestGeo3DPoint
[junit4] 2> ??? 11, 2017 5:01:04 ? com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
[junit4] 2> WARNING: Suite execution timed out: org.apache.lucene.spatial3d.TestGeo3DPoint
[junit4] 2> 1) Thread[id=1, name=main, state=WAITING, group=main]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Thread.join(Thread.java:1252)
[junit4] 2> at java.lang.Thread.join(Thread.java:1326)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:608)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:457)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:251)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:368)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:13)
[junit4] 2> 2) Thread[id=9, name=JUnit4-serializer-daemon, state=TIMED_WAITING, group=main]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:50)
[junit4] 2> 3) Thread[id=13, name=SUITE-TestGeo3DPoint-seed#[21D6FA9DD0AC0AF2], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at java.lang.Thread.getStackTrace(Thread.java:1559)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:696)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:693)
[junit4] 2> at java.security.AccessController.doPrivileged(Native Method)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getStackTrace(ThreadLeakControl.java:693)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getThreadsWithTraces(ThreadLeakControl.java:709)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:689)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:65)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:415)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:678)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:140)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:598)
[junit4] 2> 4) Thread[id=14, name=TEST-TestGeo3DPoint.testRandomMedium-seed#[21D6FA9DD0AC0AF2], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at org.apache.lucene.spatial3d.geom.Plane.findIntersectionBounds(Plane.java:1053)
[junit4] 2> at org.apache.lucene.spatial3d.geom.Plane.recordBounds(Plane.java:1214)
[junit4] 2> at org.apache.lucene.spatial3d.geom.XYZBounds.addIntersection(XYZBounds.java:262)
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoStandardPath$PathSegment.getBounds(GeoStandardPath.java:851)
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoStandardPath.getBounds(GeoStandardPath.java:315)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.verify(TestGeo3DPoint.java:846)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.doTestRandom(TestGeo3DPoint.java:548)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomMedium(TestGeo3DPoint.java:475)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestGeo3DPoint -Dtests.method=testRandomMedium -Dtests.seed=21D6FA9DD0AC0AF2 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=ar-OM -Dtests.timezone=America/North_Dakota/New_Salem -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 18.6s J1 | TestGeo3DPoint.testRandomMedium <<<
[junit4] > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
[junit4] > at __randomizedtesting.SeedInfo.seed([21D6FA9DD0AC0AF2]:0)
[junit4] 2> ??? 11, 2017 5:01:04 ? com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> ??? 11, 2017 5:01:25 ? com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.lucene.spatial3d.TestGeo3DPoint:
[junit4] 2> 1) Thread[id=14, name=TEST-TestGeo3DPoint.testRandomMedium-seed#[21D6FA9DD0AC0AF2], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at org.apache.lucene.spatial3d.geom.Plane.addPoint(Plane.java:1953)
[junit4] 2> at org.apache.lucene.spatial3d.geom.Plane.recordBounds(Plane.java:1591)
[junit4] 2> at org.apache.lucene.spatial3d.geom.XYZBounds.addPlane(XYZBounds.java:161)
[junit4] 2> at org.apache.lucene.spatial3d.geom.XYZBounds.addVerticalPlane(XYZBounds.java:194)
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoRectangle.getBounds(GeoRectangle.java:213)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.verify(TestGeo3DPoint.java:846)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.doTestRandom(TestGeo3DPoint.java:548)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomMedium(TestGeo3DPoint.java:475)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> ??? 11, 2017 5:01:25 ? com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=14, name=TEST-TestGeo3DPoint.testRandomMedium-seed#[21D6FA9DD0AC0AF2], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> ??? 11, 2017 5:01:28 ? com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> SEVERE: There are still zombie threads that couldn't be terminated:
[junit4] 2> 1) Thread[id=14, name=TEST-TestGeo3DPoint.testRandomMedium-seed#[21D6FA9DD0AC0AF2], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at org.apache.lucene.spatial3d.geom.Plane.addPoint(Plane.java:1953)
[junit4] 2> at org.apache.lucene.spatial3d.geom.Plane.recordBounds(Plane.java:1418)
[junit4] 2> at org.apache.lucene.spatial3d.geom.XYZBounds.addPlane(XYZBounds.java:161)
[junit4] 2> at org.apache.lucene.spatial3d.geom.XYZBounds.addVerticalPlane(XYZBounds.java:194)
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoRectangle.getBounds(GeoRectangle.java:211)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.verify(TestGeo3DPoint.java:846)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.doTestRandom(TestGeo3DPoint.java:548)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomMedium(TestGeo3DPoint.java:475)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarity(queryNorm=true): {}, locale=ar-OM, timezone=America/North_Dakota/New_Salem
[junit4] 2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=2,free=238835584,total=505413632
[junit4] 2> NOTE: All tests run in this JVM: [XYZSolidTest, TestGeo3DPoint]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestGeo3DPoint -Dtests.seed=21D6FA9DD0AC0AF2 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=ar-OM -Dtests.timezone=America/North_Dakota/New_Salem -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 0.00s J1 | TestGeo3DPoint (suite) <<<
[junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4] > at __randomizedtesting.SeedInfo.seed([21D6FA9DD0AC0AF2]:0)
[junit4] Completed [14/14 (1!)] on J1 in 7223.53s, 9 tests, 2 errors <<< FAILURES!
[...truncated 1894 lines...]
[junit4] Suite: org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_D656E80D4531B37C-001/init-core-data-001
[junit4] 2> 1413219 WARN (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[D656E80D4531B37C]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
[junit4] 2> 1413275 INFO (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[D656E80D4531B37C]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 1413276 INFO (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[D656E80D4531B37C]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 1413276 INFO (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[D656E80D4531B37C]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 1413276 INFO (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[D656E80D4531B37C]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /hr/
[junit4] 2> 1416154 WARN (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[D656E80D4531B37C]-worker) [ ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 1420305 WARN (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[D656E80D4531B37C]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 1421315 INFO (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[D656E80D4531B37C]-worker) [ ] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
[junit4] 2> 1421399 WARN (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[D656E80D4531B37C]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 1422293 INFO (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[D656E80D4531B37C]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 1422524 INFO (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[D656E80D4531B37C]-worker) [ ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/hdfs to ./temp/Jetty_lucene2.us.west_apache_org_36918_hdfs____kiqnuv/webapp
[junit4] 2> 1425254 INFO (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[D656E80D4531B37C]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@lucene2-us-west.apache.org:36918
[junit4] 2> 1431460 WARN (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[D656E80D4531B37C]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 1431485 INFO (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[D656E80D4531B37C]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 1431578 INFO (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[D656E80D4531B37C]-worker) [ ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_44614_datanode____u5orsw/webapp
[junit4] 2> 1432746 INFO (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[D656E80D4531B37C]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44614
[junit4] 2> 1433928 WARN (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[D656E80D4531B37C]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 1433929 INFO (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[D656E80D4531B37C]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 1434007 INFO (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[D656E80D4531B37C]-worker) [ ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_46847_datanode____nhub1c/webapp
[junit4] 2> 1435490 INFO (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[D656E80D4531B37C]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:46847
[junit4] 2> 1442936 INFO (IPC Server handler 4 on 44513) [ ] BlockStateChange BLOCK* processReport: from storage DS-2ae2b83b-44d8-44a7-8e1e-ab98dd8eb8ec node DatanodeRegistration(127.0.0.1:35508, datanodeUuid=7f2a5dff-4c00-4fe0-abf9-f774caeb0a25, infoPort=40770, infoSecurePort=0, ipcPort=43409, storageInfo=lv=-56;cid=testClusterID;nsid=301380055;c=0), blocks: 0, hasStaleStorage: true, processing time: 28 msecs
[junit4] 2> 1442953 INFO (IPC Server handler 4 on 44513) [ ] BlockStateChange BLOCK* processReport: from storage DS-839c2a56-4784-470a-97d0-16977988b289 node DatanodeRegistration(127.0.0.1:35508, datanodeUuid=7f2a5dff-4c00-4fe0-abf9-f774caeb0a25, infoPort=40770, infoSecurePort=0, ipcPort=43409, storageInfo=lv=-56;cid=testClusterID;nsid=301380055;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 1443119 INFO (IPC Server handler 3 on 44513) [ ] BlockStateChange BLOCK* processReport: from storage DS-1f6b9a22-f28f-439c-a5fe-53dd21349e1b node DatanodeRegistration(127.0.0.1:35124, datanodeUuid=28db76e7-4728-4e7b-8ed5-8ddae8e899d4, infoPort=34561, infoSecurePort=0, ipcPort=43641, storageInfo=lv=-56;cid=testClusterID;nsid=301380055;c=0), blocks: 0, hasStaleStorage: true, processing time: 134 msecs
[junit4] 2> 1443119 INFO (IPC Server handler 3 on 44513) [ ] BlockStateChange BLOCK* processReport: from storage DS-d711034b-03b3-4345-b754-84fc33bc4806 node DatanodeRegistration(127.0.0.1:35124, datanodeUuid=28db76e7-4728-4e7b-8ed5-8ddae8e899d4, infoPort=34561, infoSecurePort=0, ipcPort=43641, storageInfo=lv=-56;cid=testClusterID;nsid=301380055;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 1444922 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1444927 INFO (Thread-4455) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1444927 INFO (Thread-4455) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1444987 ERROR (Thread-4455) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1445041 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.a.s.c.ZkTestServer start zk server on port:42970
[junit4] 2> 1445707 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 1445724 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 1445725 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1445726 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 1445727 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 1445728 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 1445749 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 1445750 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 1445751 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1445752 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 1445766 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 1445814 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
[junit4] 2> 1446773 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1446775 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@183ba84e{/hr,null,AVAILABLE}
[junit4] 2> 1446775 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@33d56e47{HTTP/1.1,[http/1.1]}{127.0.0.1:38610}
[junit4] 2> 1446775 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.e.j.s.Server Started @1457888ms
[junit4] 2> 1446775 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://lucene2-us-west.apache.org:44513/hdfs__lucene2-us-west.apache.org_44513__home_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-7.x_checkout_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_D656E80D4531B37C-001_tempDir-002_control_data, hostContext=/hr, hostPort=38610, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_D656E80D4531B37C-001/control-001/cores}
[junit4] 2> 1446775 ERROR (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1446776 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.1.0
[junit4] 2> 1446776 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1446776 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 1446776 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-11T10:31:25.481Z
[junit4] 2> 1446984 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1446984 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_D656E80D4531B37C-001/control-001/solr.xml
[junit4] 2> 1447020 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15dd0d93b010002, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 1447053 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@48266f1b, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 1447147 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42970/solr
[junit4] 2> 1448753 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [n:127.0.0.1:38610_hr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1448754 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [n:127.0.0.1:38610_hr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:38610_hr
[junit4] 2> 1448754 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [n:127.0.0.1:38610_hr ] o.a.s.c.Overseer Overseer (id=98464398288420868-127.0.0.1:38610_hr-n_0000000000) starting
[junit4] 2> 1448967 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [n:127.0.0.1:38610_hr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38610_hr
[junit4] 2> 1448968 INFO (zkCallback-187-thread-1-processing-n:127.0.0.1:38610_hr) [n:127.0.0.1:38610_hr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1449636 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [n:127.0.0.1:38610_hr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@48266f1b
[junit4] 2> 1449668 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [n:127.0.0.1:38610_hr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@48266f1b
[junit4] 2> 1449668 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [n:127.0.0.1:38610_hr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@48266f1b
[junit4] 2> 1449683 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [n:127.0.0.1:38610_hr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_D656E80D4531B37C-001/control-001/cores
[junit4] 2> 1450019 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1450019 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42970/solr ready
[junit4] 2> 1450049 INFO (qtp1780321128-5756) [n:127.0.0.1:38610_hr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:38610_hr&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1450160 INFO (OverseerThreadFactory-609-thread-1-processing-n:127.0.0.1:38610_hr) [n:127.0.0.1:38610_hr ] o.a.s.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 1450499 INFO (qtp1780321128-5754) [n:127.0.0.1:38610_hr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1450499 INFO (qtp1780321128-5754) [n:127.0.0.1:38610_hr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 1450965 INFO (zkCallback-187-thread-1-processing-n:127.0.0.1:38610_hr) [n:127.0.0.1:38610_hr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1451862 INFO (qtp1780321128-5754) [n:127.0.0.1:38610_hr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 1452064 INFO (qtp1780321128-5754) [n:127.0.0.1:38610_hr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
[junit4] 2> 1452704 INFO (qtp1780321128-5754) [n:127.0.0.1:38610_hr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 1452853 INFO (qtp1780321128-5754) [n:127.0.0.1:38610_hr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
[junit4] 2> 1452868 INFO (qtp1780321128-5754) [n:127.0.0.1:38610_hr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@48266f1b
[junit4] 2> 1452963 INFO (qtp1780321128-5754) [n:127.0.0.1:38610_hr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://lucene2-us-west.apache.org:44513/solr_hdfs_home
[junit4] 2> 1452965 INFO (qtp1780321128-5754) [n:127.0.0.1:38610_hr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 1452965 INFO (qtp1780321128-5754) [n:127.0.0.1:38610_hr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1452965 INFO (qtp1780321128-5754) [n:127.0.0.1:38610_hr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_D656E80D4531B37C-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[hdfs://lucene2-us-west.apache.org:44513/solr_hdfs_home/control_collection/core_node2/data/]
[junit4] 2> 1452966 INFO (qtp1780321128-5754) [n:127.0.0.1:38610_hr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:44513/solr_hdfs_home/control_collection/core_node2/data/snapshot_metadata
[junit4] 2> 1453124 INFO (qtp1780321128-5754) [n:127.0.0.1:38610_hr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1453124 INFO (qtp1780321128-5754) [n:127.0.0.1:38610_hr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 1453124 INFO (qtp1780321128-5754) [n:127.0.0.1:38610_hr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Creating new global HDFS BlockCache
[junit4] 2> 1457679 INFO (qtp1780321128-5754) [n:127.0.0.1:38610_hr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1457722 INFO (qtp1780321128-5754) [n:127.0.0.1:38610_hr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:44513/solr_hdfs_home/control_collection/core_node2/data
[junit4] 2> 1458536 INFO (qtp1780321128-5754) [n:127.0.0.1:38610_hr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:44513/solr_hdfs_home/control_collection/core_node2/data/index
[junit4] 2> 1458601 INFO (qtp1780321128-5754) [n:127.0.0.1:38610_hr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1458601 INFO (qtp1780321128-5754) [n:127.0.0.1:38610_hr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 1458680 INFO (qtp1780321128-5754) [n:127.0.0.1:38610_hr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1458681 INFO (qtp1780321128-5754) [n:127.0.0.1:38610_hr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1345021910, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 1460715 INFO (IPC Server handler 8 on 44513) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:35508 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-d711034b-03b3-4345-b754-84fc33bc4806:NORMAL:127.0.0.1:35124|RBW], ReplicaUC[[DISK]DS-2ae2b83b-44d8-44a7-8e1e-ab98dd8eb8ec:NORMAL:127.0.0.1:35508|RBW]]} size 0
[junit4] 2> 1460827 INFO (IPC Server handler 0 on 44513) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:35124 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-d711034b-03b3-4345-b754-84fc33bc4806:NORMAL:127.0.0.1:35124|RBW], ReplicaUC[[DISK]DS-2ae2b83b-44d8-44a7-8e1e-ab98dd8eb8ec:NORMAL:127.0.0.1:35508|RBW]]} size 0
[junit4] 2> 1461042 WARN (qtp1780321128-5754) [n:127.0.0.1:38610_hr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1461567 INFO (qtp1780321128-5754) [n:127.0.0.1:38610_hr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 1461568 INFO (qtp1780321128-5754) [n:127.0.0.1:38610_hr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1461568 INFO (qtp1780321128-5754) [n:127.0.0.1:38610_hr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 1461760 INFO (qtp1780321128-5754) [n:127.0.0.1:38610_hr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 15000ms;
[junit4] 2> 1461760 INFO (qtp1780321128-5754) [n:127.0.0.1:38610_hr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1461792 INFO (qtp1780321128-5754) [n:127.0.0.1:38610_hr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=23, maxMergeAtOnceExplicit=26, maxMergedSegmentMB=50.564453125, floorSegmentMB=0.7978515625, forceMergeDeletesPctAllowed=20.63886012023828, segmentsPerTier=46.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.21095445543267355
[junit4] 2> 1462464 INFO (qtp1780321128-5754) [n:127.0.0.1:38610_hr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2cc32e90[control_collection_shard1_replica_n1] main]
[junit4] 2> 1462466 INFO (qtp1780321128-5754) [n:127.0.0.1:38610_hr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1462466 INFO (qtp1780321128-5754) [n:127.0.0.1:38610_hr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1462511 INFO (qtp1780321128-5754) [n:127.0.0.1:38610_hr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1462513 INFO (qtp1780321128-5754) [n:127.0.0.1:38610_hr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575430391037165568
[junit4] 2> 1462534 INFO (searcherExecutor-612-thread-1-processing-n:127.0.0.1:38610_hr x:control_collection_shard1_replica_n1 s:shard1 c:control_collection) [n:127.0.0.1:38610_hr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@2cc32e90[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1462550 INFO (qtp1780321128-5754) [n:127.0.0.1:38610_hr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1462550 INFO (qtp1780321128-5754) [n:127.0.0.1:38610_hr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1462550 INFO (qtp1780321128-5754) [n:127.0.0.1:38610_hr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:38610/hr/control_collection_shard1_replica_n1/
[junit4] 2> 1462550 INFO (qtp1780321128-5754) [n:127.0.0.1:38610_hr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1462550 INFO (qtp1780321128-5754) [n:127.0.0.1:38610_hr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:38610/hr/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 1462550 INFO (qtp1780321128-5754) [n:127.0.0.1:38610_hr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 1462566 INFO (qtp1780321128-5754) [n:127.0.0.1:38610_hr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:38610/hr/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 1462594 INFO (zkCallback-187-thread-1-processing-n:127.0.0.1:38610_hr) [n:127.0.0.1:38610_hr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1462857 INFO (zkCallback-187-thread-1-processing-n:127.0.0.1:38610_hr) [n:127.0.0.1:38610_hr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1462921 INFO (qtp1780321128-5754) [n:127.0.0.1:38610_hr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1462922 INFO (qtp1780321128-5754) [n:127.0.0.1:38610_hr c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=12423
[junit4] 2> 1463096 INFO (zkCallback-187-thread-1-processing-n:127.0.0.1:38610_hr) [n:127.0.0.1:38610_hr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1463099 INFO (qtp1780321128-5756) [n:127.0.0.1:38610_hr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 1463099 INFO (qtp1780321128-5756) [n:127.0.0.1:38610_hr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:38610_hr&wt=javabin&version=2} status=0 QTime=13049
[junit4] 2> 1463194 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1463195 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42970/solr ready
[junit4] 2> 1463197 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 1463228 INFO (qtp1780321128-5757) [n:127.0.0.1:38610_hr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&stateFormat=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1463243 INFO (OverseerThreadFactory-609-thread-2-processing-n:127.0.0.1:38610_hr) [n:127.0.0.1:38610_hr ] o.a.s.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 1463244 INFO (OverseerCollectionConfigSetProcessor-98464398288420868-127.0.0.1:38610_hr-n_0000000000) [n:127.0.0.1:38610_hr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1463244 WARN (OverseerThreadFactory-609-thread-2-processing-n:127.0.0.1:38610_hr) [n:127.0.0.1:38610_hr ] o.a.s.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
[junit4] 2> 1463527 INFO (qtp1780321128-5757) [n:127.0.0.1:38610_hr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 1463527 INFO (qtp1780321128-5757) [n:127.0.0.1:38610_hr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&stateFormat=1&wt=javabin&version=2} status=0 QTime=301
[junit4] 2> 1464334 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_D656E80D4531B37C-001/shard-1-001 of type NRT
[junit4] 2> 1464335 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1464437 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6a7f4d25{/hr,null,AVAILABLE}
[junit4] 2> 1464437 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@4ce903b9{HTTP/1.1,[http/1.1]}{127.0.0.1:43068}
[junit4] 2> 1464437 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.e.j.s.Server Started @1475550ms
[junit4] 2> 1464438 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://lucene2-us-west.apache.org:44513/hdfs__lucene2-us-west.apache.org_44513__home_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-7.x_checkout_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_D656E80D4531B37C-001_tempDir-002_jetty1, solrconfig=solrconfig.xml, hostContext=/hr, hostPort=43068, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_D656E80D4531B37C-001/shard-1-001/cores}
[junit4] 2> 1464438 ERROR (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1464438 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.1.0
[junit4] 2> 1464438 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1464438 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 1464438 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-11T10:31:43.143Z
[junit4] 2> 1464493 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1464493 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_D656E80D4531B37C-001/shard-1-001/solr.xml
[junit4] 2> 1464543 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@48266f1b, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 1464588 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42970/solr
[junit4] 2> 1465041 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [n:127.0.0.1:43068_hr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1465054 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [n:127.0.0.1:43068_hr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1465055 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [n:127.0.0.1:43068_hr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43068_hr
[junit4] 2> 1465057 INFO (zkCallback-194-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1465057 INFO (zkCallback-187-thread-1-processing-n:127.0.0.1:38610_hr) [n:127.0.0.1:38610_hr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1465214 INFO (zkCallback-199-thread-1-processing-n:127.0.0.1:43068_hr) [n:127.0.0.1:43068_hr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1465249 INFO (OverseerCollectionConfigSetProcessor-98464398288420868-127.0.0.1:38610_hr-n_0000000000) [n:127.0.0.1:38610_hr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1465580 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [n:127.0.0.1:43068_hr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@48266f1b
[junit4] 2> 1465612 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [n:127.0.0.1:43068_hr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@48266f1b
[junit4] 2> 1465612 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [n:127.0.0.1:43068_hr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@48266f1b
[junit4] 2> 1465626 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[D656E80D4531B37C]) [n:127.0.0.1:43068_hr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_D656E80D4531B37C-001/shard-1-001/cores
[junit4] 2> 1466026 INFO (qtp1780321128-5758) [n:127.0.0.1:38610_hr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:43068_hr&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1466059 INFO (OverseerThreadFactory-609-thread-3-processing-n:127.0.0.1:38610_hr) [n:127.0.0.1:38610_hr ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:43068_hr for creating new replica
[junit4] 2> 1466154 INFO (qtp1901062972-5807) [n:127.0.0.1:43068_hr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n21&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1466155 INFO (qtp1901062972-5807) [n:127.0.0.1:43068_hr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 1467301 INFO (qtp1901062972-5807) [n:127.0.0.1:43068_hr c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 1467450 INFO (qtp1901062972-5807) [n:127.0.0.1:43068_hr c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.s.IndexSchema [collection1_shard1_replica_n21] Schema name=test
[junit4] 2> 1468139 INFO (qtp1901062972-5807) [n:127.0.0.1:43068_hr c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 1468236 INFO (qtp1901062972-5807) [n:127.0.0.1:43068_hr c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n21' using configuration from collection collection1, trusted=true
[junit4] 2> 1468236 INFO (qtp1901062972-5807) [n:127.0.0.1:43068_hr c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n21' (registry 'solr.core.collection1.shard1.replica_n21') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@48266f1b
[junit4] 2> 1468236 INFO (qtp1901062972-5807) [n:127.0.0.1:43068_hr c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://lucene2-us-west.apache.org:44513/solr_hdfs_home
[junit4] 2> 1468236 INFO (qtp1901062972-5807) [n:127.0.0.1:43068_hr c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 1468236 INFO (qtp1901062972-5807) [n:127.0.0.1:43068_hr c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1468236 INFO (qtp1901062972-5807) [n:127.0.0.1:43068_hr c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SolrCore [[collection1_shard1_replica_n21] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_D656E80D4531B37C-001/shard-1-001/cores/collection1_shard1_replica_n21], dataDir=[hdfs://lucene2-us-west.apache.org:44513/solr_hdfs_home/collection1/core_node22/data/]
[junit4] 2> 1468266 INFO (qtp1901062972-5807) [n:127.0.0.1:43068_hr c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:44513/solr_hdfs_home/collection1/core_node22/data/snapshot_metadata
[junit4] 2> 1468331 INFO (qtp1901062972-5807) [n:127.0.0.1:43068_hr c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1468331 INFO (qtp1901062972-5807) [n:127.0.0.1:43068_hr c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 1468606 INFO (qtp1901062972-5807) [n:127.0.0.1:43068_hr c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1468623 INFO (qtp1901062972-5807) [n:127.0.0.1:43068_hr c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:44513/solr_hdfs_home/collection1/core_node22/data
[junit4] 2> 1468836 INFO (qtp1901062972-5807) [n:127.0.0.1:43068_hr c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:44513/solr_hdfs_home/collection1/core_node22/data/index
[junit4] 2> 1468883 INFO (qtp1901062972-5807) [n:127.0.0.1:43068_hr c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1468883 INFO (qtp1901062972-5807) [n:127.0.0.1:43068_hr c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 1468919 INFO (qtp1901062972-5807) [n:127.0.0.1:43068_hr c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1468921 INFO (qtp1901062972-5807) [n:127.0.0.1:43068_hr c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1345021910, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 1469472 INFO (IPC Server handler 5 on 44513) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:35124 is added to blk_1073741826_1002{UCState=COMMITTED, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-839c2a56-4784-470a-97d0-16977988b289:NORMAL:127.0.0.1:35508|RBW], ReplicaUC[[DISK]DS-d711034b-03b3-4345-b754-84fc33bc4806:NORMAL:127.0.0.1:35124|RBW]]} size 72
[junit4] 2> 1469473 INFO (IPC Server handler 1 on 44513) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:35508 is added to blk_1073741826_1002 size 72
[junit4] 2> 1469958 WARN (qtp1901062972-5807) [n:127.0.0.1:43068_hr c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1470304 INFO (qtp1901062972-5807) [n:127.0.0.1:43068_hr c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 1470304 INFO (qtp1901062972-5807) [n:127.0.0.1:43068_hr c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1470304 INFO (qtp1901062972-5807) [n:127.0.0.1:43068_hr c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 1470386 INFO (qtp1901062972-5807) [n:127.0.0.1:43068_hr c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 15000ms;
[junit4] 2> 1470387 INFO (qtp1901062972-5807) [n:127.0.0.1:43068_hr c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1470403 INFO (qtp1901062972-5807) [n:127.0.0.1:43068_hr c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=23, maxMergeAtOnceExplicit=26, maxMergedSegmentMB=50.564453125, floorSegmentMB=0.7978515625, forceMergeDeletesPctAllowed=20.63886012023828, segmentsPerTier=46.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.21095445543267355
[junit4] 2> 1470610 INFO (qtp1901062972-5807) [n:127.0.0.1:43068_hr c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.s.SolrIndexSearcher Opening [Searcher@620eab52[collection1_shard1_replica_n21] main]
[junit4] 2> 1470610 INFO (qtp1901062972-5807) [n:127.0.0.1:43068_hr c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1470611 INFO (qtp1901062972-5807) [n:127.0.0.1:43068_hr c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1470611 INFO (qtp1901062972-5807) [n:127.0.0.1:43068_hr c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1470630 INFO (searcherExecutor-623-thread-1-processing-n:127.0.0.1:43068_hr x:collection1_shard1_replica_n21 s:shard1 c:collection1) [n:127.0.0.1:43068_hr c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SolrCore [collection1_shard1_replica_n21] Registered new searcher Searcher@620eab52[collection1_shard1_replica_n21] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1470630 INFO (qtp1901062972-5807) [n:127.0.0.1:43068_hr c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575430399548456960
[junit4] 2> 1470691 INFO (qtp1901062972-5807) [n:127.0.0.1:43068_hr c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1470691 INFO (qtp1901062972-5807) [n:127.0.0.1:43068_hr c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1470692 INFO (qtp1901062972-5807) [n:127.0.0.1:43068_hr c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:43068/hr/collection1_shard1_replica_n21/
[junit4] 2> 1470692 INFO (qtp1901062972-5807) [n:127.0.0.1:43068_hr c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SyncStrategy Sync Success -
[...truncated too long message...]
2_replica_n8] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.localShardsTestColl.shard2.leader, tag=1469018633
[junit4] 2> 151359 INFO (coreCloseExecutor-163-thread-1) [n:127.0.0.1:36120_solr c:localShardsTestColl s:shard1 r:core_node5 x:localShardsTestColl_shard1_replica_n3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.localShardsTestColl.shard1.replica_n3, tag=2016005557
[junit4] 2> 151361 INFO (coreCloseExecutor-165-thread-2) [n:127.0.0.1:39320_solr c:localShardsTestColl s:shard2 r:core_node9 x:localShardsTestColl_shard2_replica_n6] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.localShardsTestColl.shard2.leader, tag=418396727
[junit4] 2> 151373 INFO (coreCloseExecutor-165-thread-4) [n:127.0.0.1:39320_solr c:localShardsTestColl s:shard1 r:core_node2 x:localShardsTestColl_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.localShardsTestColl.shard1.replica_n1, tag=335577197
[junit4] 2> 151375 INFO (coreCloseExecutor-165-thread-4) [n:127.0.0.1:39320_solr c:localShardsTestColl s:shard1 r:core_node2 x:localShardsTestColl_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.localShardsTestColl.shard1.leader, tag=335577197
[junit4] 2> 151375 INFO (coreCloseExecutor-165-thread-5) [n:127.0.0.1:39320_solr c:2nd_collection s:shard2 r:core_node4 x:2nd_collection_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.2nd_collection.shard2.replica_n2, tag=1597737755
[junit4] 2> 151426 INFO (coreCloseExecutor-164-thread-2) [n:127.0.0.1:33073_solr c:localShardsTestColl s:shard2 r:core_node13 x:localShardsTestColl_shard2_replica_n10] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.localShardsTestColl.shard2.leader, tag=1391408747
[junit4] 2> 151426 INFO (coreCloseExecutor-164-thread-3) [n:127.0.0.1:33073_solr c:localShardsTestColl s:shard3 r:core_node18 x:localShardsTestColl_shard3_replica_n16] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.localShardsTestColl.shard3.leader, tag=1124421342
[junit4] 2> 151438 INFO (coreCloseExecutor-164-thread-6) [n:127.0.0.1:33073_solr c:multicollection1 s:shard1 r:core_node3 x:multicollection1_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.multicollection1.shard1.replica_n1, tag=261454689
[junit4] 2> 151440 INFO (coreCloseExecutor-164-thread-6) [n:127.0.0.1:33073_solr c:multicollection1 s:shard1 r:core_node3 x:multicollection1_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.multicollection1.shard1.leader, tag=261454689
[junit4] 2> 151442 INFO (coreCloseExecutor-165-thread-1) [n:127.0.0.1:39320_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard2.leader, tag=1103047782
[junit4] 2> 151442 INFO (coreCloseExecutor-165-thread-8) [n:127.0.0.1:39320_solr c:multicollection2 s:shard1 r:core_node2 x:multicollection2_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.multicollection2.shard1.replica_n1, tag=1505793518
[junit4] 2> 151444 INFO (coreCloseExecutor-165-thread-8) [n:127.0.0.1:39320_solr c:multicollection2 s:shard1 r:core_node2 x:multicollection2_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.multicollection2.shard1.leader, tag=1505793518
[junit4] 2> 151444 INFO (coreCloseExecutor-165-thread-7) [n:127.0.0.1:39320_solr c:multicollection1 s:shard2 r:core_node4 x:multicollection1_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.multicollection1.shard2.replica_n2, tag=590242257
[junit4] 2> 151454 INFO (coreCloseExecutor-163-thread-1) [n:127.0.0.1:36120_solr c:localShardsTestColl s:shard1 r:core_node5 x:localShardsTestColl_shard1_replica_n3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.localShardsTestColl.shard1.leader, tag=2016005557
[junit4] 2> 151458 INFO (coreCloseExecutor-163-thread-7) [n:127.0.0.1:36120_solr c:multicollection2 s:shard2 r:core_node4 x:multicollection2_shard2_replica_n3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.multicollection2.shard2.replica_n3, tag=185195669
[junit4] 2> 151460 INFO (coreCloseExecutor-163-thread-7) [n:127.0.0.1:36120_solr c:multicollection2 s:shard2 r:core_node4 x:multicollection2_shard2_replica_n3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.multicollection2.shard2.leader, tag=185195669
[junit4] 2> 151620 INFO (jetty-closer-4-thread-2) [ ] o.a.s.c.Overseer Overseer (id=98464599905861641-127.0.0.1:36120_solr-n_0000000000) closing
[junit4] 2> 151620 INFO (coreCloseExecutor-165-thread-7) [n:127.0.0.1:39320_solr c:multicollection1 s:shard2 r:core_node4 x:multicollection1_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.multicollection1.shard2.leader, tag=590242257
[junit4] 2> 151621 INFO (coreCloseExecutor-165-thread-3) [n:127.0.0.1:39320_solr c:localShardsTestColl s:shard3 r:core_node15 x:localShardsTestColl_shard3_replica_n12] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.localShardsTestColl.shard3.leader, tag=928793314
[junit4] 2> 151632 INFO (OverseerStateUpdate-98464599905861641-127.0.0.1:36120_solr-n_0000000000) [n:127.0.0.1:36120_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:36120_solr
[junit4] 2> 151636 INFO (coreCloseExecutor-165-thread-5) [n:127.0.0.1:39320_solr c:2nd_collection s:shard2 r:core_node4 x:2nd_collection_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.2nd_collection.shard2.leader, tag=1597737755
[junit4] 2> 151738 INFO (zkCallback-18-thread-7-processing-n:127.0.0.1:39320_solr) [n:127.0.0.1:39320_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 151820 INFO (zkCallback-20-thread-3-processing-n:127.0.0.1:36120_solr) [n:127.0.0.1:36120_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 151913 INFO (jetty-closer-4-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@6234efd9{/solr,null,UNAVAILABLE}
[junit4] 2> 151914 INFO (jetty-closer-4-thread-3) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@22c7c96b{/solr,null,UNAVAILABLE}
[junit4] 2> 151929 INFO (jetty-closer-4-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@7ac3a4a8{/solr,null,UNAVAILABLE}
[junit4] 2> 152059 ERROR (SUITE-CloudSolrClientTest-seed#[67712A0B1DEDE8]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 152060 INFO (SUITE-CloudSolrClientTest-seed#[67712A0B1DEDE8]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:36645 36645
[junit4] 2> 152250 INFO (Thread-2) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:36645 36645
[junit4] 2> 152398 WARN (Thread-2) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 8 /solr/aliases.json
[junit4] 2> 3 /solr/security.json
[junit4] 2> 3 /solr/configs/conf
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 30 /solr/collections/foo/state.json
[junit4] 2> 27 /solr/collections/localShardsTestColl/state.json
[junit4] 2> 18 /solr/collections/2nd_collection/state.json
[junit4] 2> 18 /solr/collections/multicollection1/state.json
[junit4] 2> 16 /solr/collections/collection1/state.json
[junit4] 2> 16 /solr/collections/multicollection2/state.json
[junit4] 2> 14 /solr/collections/nemesis/state.json
[junit4] 2> 8 /solr/clusterprops.json
[junit4] 2> 8 /solr/clusterstate.json
[junit4] 2> 7 /solr/collections/overwrite/state.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 8 /solr/live_nodes
[junit4] 2> 8 /solr/collections
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-solrj/test/J1/temp/solr.client.solrj.impl.CloudSolrClientTest_67712A0B1DEDE8-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {multiDefault=PostingsFormat(name=LuceneFixedGap), a_t=PostingsFormat(name=LuceneFixedGap), title_s=PostingsFormat(name=LuceneFixedGap), id=Lucene50(blocksize=128), text=BlockTreeOrds(blocksize=128)}, docValues:{_version_=DocValuesFormat(name=Lucene70), multiDefault=DocValuesFormat(name=Lucene70), title_s=DocValuesFormat(name=Lucene70), intDefault=DocValuesFormat(name=Lucene70), id=DocValuesFormat(name=Asserting), timestamp=DocValuesFormat(name=Lucene70)}, maxPointsInLeafNode=53, maxMBSortInHeap=6.185994397191079, sim=RandomSimilarity(queryNorm=false): {}, locale=hu-HU, timezone=Antarctica/Davis
[junit4] 2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=123127840,total=326107136
[junit4] 2> NOTE: All tests run in this JVM: [TestToleratedUpdateError, AddEvaluatorTest, CeilingEvaluatorTest, TestSolrProperties, DivideEvaluatorTest, SolrQueryTest, TestConfigSetAdminRequest, CloudSolrClientTest]
[junit4] Completed [26/135 (1!)] on J1 in 129.21s, 15 tests, 1 error <<< FAILURES!
[...truncated 42808 lines...]
[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 50 - Still
unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/50/
13 tests failed.
FAILED: org.apache.lucene.index.TestIndexSorting.testRandom3
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([DB028E1A36FB4CA8]:0)
FAILED: junit.framework.TestSuite.org.apache.lucene.index.TestIndexSorting
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([DB028E1A36FB4CA8]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation
Error Message:
2 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation: 1) Thread[id=116587, name=jetty-launcher-27798-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037) at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328) at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277) at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323) at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105) at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41) at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244) at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44) at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61) at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505) 2) Thread[id=116581, name=jetty-launcher-27798-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037) at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328) at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277) at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323) at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105) at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41) at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244) at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44) at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61) at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:
1) Thread[id=116587, name=jetty-launcher-27798-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
2) Thread[id=116581, name=jetty-launcher-27798-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
at __randomizedtesting.SeedInfo.seed([D968CCC2A4A6BE2C]:0)
FAILED: org.apache.solr.cloud.TriLevelCompositeIdRoutingTest.test
Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:34391/_c
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:34391/_c
at __randomizedtesting.SeedInfo.seed([D968CCC2A4A6BE2C:513CF3180A5AD3D4]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:638)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1121)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:862)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:793)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createJettys(AbstractFullDistribZkTestBase.java:418)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:334)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:991)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:526)
... 44 more
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TriLevelCompositeIdRoutingTest
Error Message:
631 threads leaked from SUITE scope at org.apache.solr.cloud.TriLevelCompositeIdRoutingTest: 1) Thread[id=90738, name=qtp420515401-90738-acceptor-0@21cc8b9e-ServerConnector@4ca8f838{HTTP/1.1,[http/1.1]}{127.0.0.1:40028}, state=RUNNABLE, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:371) at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=91198, name=zkCallback-2136-thread-3, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=91252, name=zkCallback-2197-thread-7, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=91147, name=zkCallback-2124-thread-4, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=91386, name=updateExecutor-2079-thread-1, state=TERMINATED, group={null group}] at (empty stack) 6) Thread[id=90536, name=qtp1668795516-90536, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 7) Thread[id=91183, name=zkCallback-2162-thread-4, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 8) Thread[id=90418, name=qtp1384606534-90418, state=RUNNABLE, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) 9) Thread[id=90621, name=qtp1923570553-90621, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 10) Thread[id=90937, name=qtp1815404617-90937, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 11) Thread[id=91380, name=recoveryExecutor-2195-thread-2, state=TERMINATED, group={null group}] at (empty stack) 12) Thread[id=91257, name=zkCallback-2211-thread-6, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 13) Thread[id=91307, name=zkCallback-2118-thread-1-processing-n:127.0.0.1:46370__c-SendThread(127.0.0.1:41774), state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060) 14) Thread[id=91173, name=zkCallback-2169-thread-2, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 15) Thread[id=91046, name=org.eclipse.jetty.server.session.HashSessionManager@b48e968Timer, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 16) Thread[id=90419, name=qtp1384606534-90419, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 17) Thread[id=90760, name=watches-2143-thread-1, state=TERMINATED, group={null group}] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 18) Thread[id=90823, name=Scheduler-198659338, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 19) Thread[id=91241, name=zkCallback-2112-thread-6, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 20) Thread[id=91250, name=zkCallback-2190-thread-6, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 21) Thread[id=90486, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 22) Thread[id=91062, name=Scheduler-863668134, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 23) Thread[id=91275, name=zkCallback-2142-thread-6-processing-n:127.0.0.1:40028__c-EventThread, state=WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501) 24) Thread[id=90499, name=watches-2089-thread-1, state=TERMINATED, group={null group}] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 25) Thread[id=90530, name=searcherExecutor-4204-thread-1, state=WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 26) Thread[id=90991, name=Thread-79928, state=WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:502) at org.apache.solr.core.CloserThread.run(CoreContainer.java:1617) 27) Thread[id=90422, name=qtp1384606534-90422, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 28) Thread[id=90741, name=qtp420515401-90741, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 29) Thread[id=90772, name=qtp1975397537-90772, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 30) Thread[id=91415, name=coreZkRegister-4220-thread-1, state=TERMINATED, group={null group}] at (empty stack) 31) Thread[id=91301, name=zkCallback-2130-thread-2-processing-n:127.0.0.1:38543__c-SendThread(127.0.0.1:41774), state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060) 32) Thread[id=90627, name=qtp1923570553-90627, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 33) Thread[id=90657, name=org.eclipse.jetty.server.session.HashSessionManager@75f46303Timer, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 34) Thread[id=90448, name=qtp511467524-90448, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 35) Thread[id=90463, name=zkCallback-2082-thread-1, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 36) Thread[id=91443, name=coreZkRegister-4396-thread-2, state=TERMINATED, group={null group}] at (empty stack) 37) Thread[id=90983, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 38) Thread[id=91043, name=qtp2101380646-91043, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 39) Thread[id=91270, name=zkCallback-2094-thread-2-processing-n:127.0.0.1:52952__c-SendThread(127.0.0.1:41774), state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060) 40) Thread[id=91467, name=zkCallback-2088-thread-7, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 41) Thread[id=91149, name=zkCallback-2118-thread-3, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 42) Thread[id=90451, name=qtp511467524-90451, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 43) Thread[id=90997, name=searcherExecutor-4369-thread-1, state=WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 44) Thread[id=90646, name=searcherExecutor-4248-thread-1, state=WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 45) Thread[id=90715, name=org.eclipse.jetty.server.session.HashSessionManager@6e9d273fTimer, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 46) Thread[id=91274, name=zkCallback-2142-thread-6-processing-n:127.0.0.1:40028__c-SendThread(127.0.0.1:41774), state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060) 47) Thread[id=91041, name=qtp2101380646-91041, state=RUNNABLE, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) 48) Thread[id=90938, name=qtp1815404617-90938-acceptor-0@678cd15e-ServerConnector@5e7d6b66{HTTP/1.1,[http/1.1]}{127.0.0.1:43634}, state=RUNNABLE, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:371) at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) 49) Thread[id=90773, name=org.eclipse.jetty.server.session.HashSessionManager@83dfd6bTimer, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 50) Thread[id=90539, name=qtp1668795516-90539, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 51) Thread[id=90594, name=qtp701330800-90594, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 52) Thread[id=91479, name=zkCallback-2082-thread-7, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 53) Thread[id=91085, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 54) Thread[id=91353, name=zkCallback-2106-thread-7, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 55) Thread[id=90925, name=Scheduler-2129282568, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 56) Thread[id=91303, name=zkCallback-2112-thread-2-processing-n:127.0.0.1:55828__c-SendThread(127.0.0.1:41774), state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1051) 57) Thread[id=91113, name=qtp538058931-91113, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 58) Thread[id=90567, name=qtp1793083745-90567, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 59) Thread[id=90769, name=qtp1975397537-90769, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 60) Thread[id=90858, name=watches-2163-thread-1, state=TERMINATED, group={null group}] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 61) Thread[id=91005, name=qtp2071205677-91005, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at java.lang.Object.wait(Native Method) at org.apache.solr.cloud.OverseerTaskQueue$LatchWatcher.await(OverseerTaskQueue.java:149) at org.apache.solr.cloud.OverseerTaskQueue.offer(OverseerTaskQueue.java:199) at org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:288) at org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:232) at org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:210) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177) at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:745) at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:726) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:507) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:493) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.eclipse.jetty.server.Server.handle(Server.java:534) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108) at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) 62) Thread[id=90755, name=zkCallback-2142-thread-1, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 63) Thread[id=90508, name=qtp796577628-90508, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 64) Thread[id=91065, name=searcherExecutor-4391-thread-1, state=WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 65) Thread[id=91142, name=zkCallback-2155-thread-3, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 66) Thread[id=91468, name=zkCallback-2176-thread-7, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 67) Thread[id=90446, name=qtp511467524-90446-acceptor-0@13471102-ServerConnector@418a42de{HTTP/1.1,[http/1.1]}{127.0.0.1:44569}, state=RUNNABLE, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:371) at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) 68) Thread[id=90894, name=searcherExecutor-4336-thread-1, state=WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 69) Thread[id=91227, name=zkCallback-2124-thread-5, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 70) Thread[id=91306, name=zkCallback-2148-thread-1-processing-n:127.0.0.1:56481__c-EventThread, state=WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501) 71) Thread[id=90875, name=qtp1547432011-90875, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 72) Thread[id=90887, name=zkCallback-2169-thread-1, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 73) Thread[id=91188, name=zkCallback-2100-thread-2, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 74) Thread[id=90477, name=qtp1315975620-90477, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 75) Thread[id=90516, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 76) Thread[id=90466, name=Scheduler-412800778, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 77) Thread[id=90559, name=searcherExecutor-4215-thread-1, state=WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 78) Thread[id=91217, name=zkCallback-2204-thread-3, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 79) Thread[id=91239, name=zkCallback-2148-thread-6, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 80) Thread[id=90685, name=qtp1042974135-90685, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 81) Thread[id=90595, name=qtp701330800-90595, state=RUNNABLE, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) 82) Thread[id=90568, name=qtp1793083745-90568, state=RUNNABLE, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) 83) Thread[id=90870, name=qtp1547432011-90870-acceptor-0@4a4331c4-ServerConnector@6eeb32c{HTTP/1.1,[http/1.1]}{127.0.0.1:60124}, state=RUNNABLE, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:371) at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) 84) Thread[id=91155, name=zkCallback-2088-thread-3, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 85) Thread[id=90979, name=org.eclipse.jetty.server.session.HashSessionManager@3d9d0519Timer, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 86) Thread[id=91259, name=zkCallback-2100-thread-5, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 87) Thread[id=90563, name=qtp1793083745-90563, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 88) Thread[id=91392, name=recoveryExecutor-2167-thread-2, state=TERMINATED, group={null group}] at (empty stack) 89) Thread[id=91196, name=zkCallback-2082-thread-3, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 90) Thread[id=90575, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 91) Thread[id=91159, name=zkCallback-2130-thread-3, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 92) Thread[id=91174, name=zkCallback-2169-thread-3, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 93) Thread[id=91009, name=qtp2071205677-91009, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 94) Thread[id=91379, name=updateExecutor-2194-thread-2, state=TERMINATED, group={null group}] at (empty stack) 95) Thread[id=90537, name=qtp1668795516-90537, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 96) Thread[id=91264, name=zkCallback-2176-thread-4-processing-n:127.0.0.1:39578__c-SendThread(127.0.0.1:41774), state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060) 97) Thread[id=91308, name=zkCallback-2118-thread-1-processing-n:127.0.0.1:46370__c-EventThread, state=WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501) 98) Thread[id=91170, name=zkCallback-2176-thread-3, state=TIMED_WAITING, group=TGRP-TriLevelCompositeIdRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupp
[...truncated too long message...]
zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
[junit4] 2> 8688625 ERROR (jetty-launcher-27798-thread-1-EventThread) [n:127.0.0.1:37301_solr ] o.a.c.f.i.CuratorFrameworkImpl Watcher exception
[junit4] 2> java.lang.InterruptedException
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1039)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
[junit4] 2> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
[junit4] 2> at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
[junit4] 2> at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
[junit4] 2> at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
[junit4] 2> Sep 25, 2017 11:41:25 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: All leaked threads terminated.
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=1059, maxMBSortInHeap=5.738102738734357, sim=RandomSimilarity(queryNorm=true): {}, locale=sq-AL, timezone=Africa/Accra
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=273705368,total=523239424
[junit4] 2> NOTE: All tests run in this JVM: [TestDocTermOrds, TestAddFieldRealTimeGet, UnloadDistributedZkTest, TestLFUCache, RegexBoostProcessorTest, TestUseDocValuesAsStored, SpatialRPTFieldTypeTest, CreateCollectionCleanupTest, TestStressReorder, SolrInfoBeanTest, ChaosMonkeySafeLeaderWithPullReplicasTest, TestSuggestSpellingConverter, CdcrBootstrapTest, TestSolrCoreProperties, TestTrie, RecoveryAfterSoftCommitTest, QueryResultKeyTest, TestStressLiveNodes, TestManagedResource, ShowFileRequestHandlerTest, TestReRankQParserPlugin, SignatureUpdateProcessorFactoryTest, TestManagedSchema, TestSolrDeletionPolicy2, QueryEqualityTest, TestBlendedInfixSuggestions, TestDefaultStatsCache, TestGeoJSONResponseWriter, LeaderElectionTest, TestTolerantSearch, BasicDistributedZk2Test, CoreAdminRequestStatusTest, TestSubQueryTransformer, SortByFunctionTest, ConcurrentDeleteAndCreateCollectionTest, TestQueryUtils, DistributedVersionInfoTest, DeleteShardTest, HdfsChaosMonkeyNothingIsSafeTest, TestSearcherReuse, ZkStateWriterTest, ReturnFieldsTest, LeaderFailureAfterFreshStartTest, PeerSyncWithIndexFingerprintCachingTest, TestFoldingMultitermQuery, URLClassifyProcessorTest, PolyFieldTest, CollectionTooManyReplicasTest, TestPivotHelperCode, TestExceedMaxTermLength, SolrMetricManagerTest, AutoCommitTest, PingRequestHandlerTest, TestCodecSupport, FastVectorHighlighterTest, AddSchemaFieldsUpdateProcessorFactoryTest, TestSolrConfigHandlerCloud, TestPolicyCloud, SolrCLIZkUtilsTest, TestPointFields, DistributedFacetPivotLargeTest, TestRecovery, TestCloudSchemaless, SearchHandlerTest, TestNonDefinedSimilarityFactory, TestTestInjection, TestQueryWrapperFilter, ActionThrottleTest, TestExclusionRuleCollectionAccess, XsltUpdateRequestHandlerTest, TestFilteredDocIdSet, HdfsTlogReplayBufferedWhileIndexingTest, FullSolrCloudDistribCmdsTest, TermsComponentTest, TestInitQParser, TestSortByMinMaxFunction, SolrShardReporterTest, DirectUpdateHandlerTest, TestZkAclsWithHadoopAuth, TestAnalyzedSuggestions, TestUtils, ClusterStateTest, TestPseudoReturnFields, DirectUpdateHandlerOptimizeTest, AnalysisErrorHandlingTest, TestCollectionAPIs, SuggesterFSTTest, HardAutoCommitTest, TestHdfsBackupRestoreCore, BufferStoreTest, TestDeleteCollectionOnDownNodes, TestCloudRecovery, TestSerializedLuceneMatchVersion, TestLeaderElectionZkExpiry, VMParamsZkACLAndCredentialsProvidersTest, ReplicationFactorTest, DistribDocExpirationUpdateProcessorTest, TestMacros, TestSimpleTextCodec, TestImplicitCoreProperties, TestBulkSchemaAPI, TimeZoneUtilsTest, BooleanFieldTest, SpatialHeatmapFacetsTest, SSLMigrationTest, TestFieldCacheWithThreads, TestSolrCloudWithSecureImpersonation]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestSolrCloudWithSecureImpersonation -Dtests.seed=D968CCC2A4A6BE2C -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=sq-AL -Dtests.timezone=Africa/Accra -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J2 | TestSolrCloudWithSecureImpersonation (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:
[junit4] > 1) Thread[id=116587, name=jetty-launcher-27798-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
[junit4] > at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
[junit4] > at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
[junit4] > at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
[junit4] > at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
[junit4] > 2) Thread[id=116581, name=jetty-launcher-27798-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
[junit4] > at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
[junit4] > at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
[junit4] > at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
[junit4] > at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
[junit4] > at __randomizedtesting.SeedInfo.seed([D968CCC2A4A6BE2C]:0)
[junit4] Completed [717/732 (9!)] on J2 in 53.51s, 12 tests, 1 error <<< FAILURES!
[...truncated 46718 lines...]
[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 49 - Still Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/49/
12 tests failed.
FAILED: org.apache.lucene.analysis.core.TestRandomChains.testRandomChainsWithLargeStrings
Error Message:
startOffset must be non-negative, and endOffset must be >= startOffset, and offsets must not go backwards startOffset=24,endOffset=31,lastStartOffset=27 for field 'dummy'
Stack Trace:
java.lang.IllegalArgumentException: startOffset must be non-negative, and endOffset must be >= startOffset, and offsets must not go backwards startOffset=24,endOffset=31,lastStartOffset=27 for field 'dummy'
at __randomizedtesting.SeedInfo.seed([5700948A30BC1917:3D5B2B9B69F239E4]:0)
at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:767)
at org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:430)
at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:392)
at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:239)
at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:481)
at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1717)
at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1462)
at org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:171)
at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:650)
at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:540)
at org.apache.lucene.analysis.core.TestRandomChains.testRandomChainsWithLargeStrings(TestRandomChains.java:883)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.lucene.search.TestInetAddressRangeQueries.testRandomBig
Error Message:
Java heap space
Stack Trace:
java.lang.OutOfMemoryError: Java heap space
at __randomizedtesting.SeedInfo.seed([FDEDA1026D94D0FC:7ABADC8DFCCDAC7C]:0)
at org.apache.lucene.util.bkd.HeapPointWriter.writePackedValue(HeapPointWriter.java:107)
at org.apache.lucene.util.bkd.HeapPointWriter.append(HeapPointWriter.java:128)
at org.apache.lucene.util.bkd.PointReader.split(PointReader.java:68)
at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1791)
at org.apache.lucene.util.bkd.BKDWriter.finish(BKDWriter.java:1008)
at org.apache.lucene.codecs.lucene60.Lucene60PointsWriter.writeField(Lucene60PointsWriter.java:130)
at org.apache.lucene.codecs.PointsWriter.mergeOneField(PointsWriter.java:62)
at org.apache.lucene.codecs.lucene60.Lucene60PointsWriter.merge(Lucene60PointsWriter.java:223)
at org.apache.lucene.index.SegmentMerger.mergePoints(SegmentMerger.java:187)
at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:136)
at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4392)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4032)
at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2235)
at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5050)
at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1720)
at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1462)
at org.apache.lucene.search.BaseRangeFieldQueryTestCase.verify(BaseRangeFieldQueryTestCase.java:190)
at org.apache.lucene.search.BaseRangeFieldQueryTestCase.doTestRandom(BaseRangeFieldQueryTestCase.java:160)
at org.apache.lucene.search.BaseRangeFieldQueryTestCase.testRandomBig(BaseRangeFieldQueryTestCase.java:75)
at org.apache.lucene.search.TestInetAddressRangeQueries.testRandomBig(TestInetAddressRangeQueries.java:81)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
FAILED: org.apache.solr.cloud.HttpPartitionTest.test
Error Message:
The partitioned replica did not get marked down expected:<[down]> but was:<[active]>
Stack Trace:
org.junit.ComparisonFailure: The partitioned replica did not get marked down expected:<[down]> but was:<[active]>
at __randomizedtesting.SeedInfo.seed([4D57C60A96F4C56B:C503F9D03808A893]:0)
at org.junit.Assert.assertEquals(Assert.java:125)
at org.apache.solr.cloud.HttpPartitionTest.testMinRf(HttpPartitionTest.java:241)
at org.apache.solr.cloud.HttpPartitionTest.test(HttpPartitionTest.java:127)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.MultiThreadedOCPTest.test
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([4D57C60A96F4C56B:C503F9D03808A893]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.cloud.MultiThreadedOCPTest.testFillWorkQueue(MultiThreadedOCPTest.java:112)
at org.apache.solr.cloud.MultiThreadedOCPTest.test(MultiThreadedOCPTest.java:67)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.hdfs.HdfsBasicDistributedZkTest.test
Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:36773
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:36773
at __randomizedtesting.SeedInfo.seed([4D57C60A96F4C56B:C503F9D03808A893]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:638)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1121)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:862)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:793)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195)
at org.apache.solr.cloud.BasicDistributedZkTest.testANewCollectionInOneInstanceWithManualShardAssignement(BasicDistributedZkTest.java:801)
at org.apache.solr.cloud.BasicDistributedZkTest.test(BasicDistributedZkTest.java:374)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:526)
... 52 more
FAILED: org.apache.solr.core.TestJmxIntegration.testJmxRegistration
Error Message:
org.apache.lucene.store.AlreadyClosedException: Already closed
Stack Trace:
javax.management.RuntimeMBeanException: org.apache.lucene.store.AlreadyClosedException: Already closed
at __randomizedtesting.SeedInfo.seed([4D57C60A96F4C56B:C386A230FBB59D0E]:0)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrow(DefaultMBeanServerInterceptor.java:839)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.rethrowMaybeMBeanException(DefaultMBeanServerInterceptor.java:852)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:651)
at com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678)
at org.apache.solr.core.TestJmxIntegration.testJmxRegistration(TestJmxIntegration.java:124)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.lucene.store.AlreadyClosedException: Already closed
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:337)
at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:349)
at org.apache.solr.core.SolrCore.getIndexDir(SolrCore.java:328)
at org.apache.solr.handler.ReplicationHandler.lambda$initializeMetrics$5(ReplicationHandler.java:873)
at org.apache.solr.metrics.reporters.jmx.JmxMetricsReporter$JmxGauge.getValue(JmxMetricsReporter.java:209)
at sun.reflect.GeneratedMethodAccessor178.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:71)
at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:275)
at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
at com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:83)
at com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:206)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:647)
... 41 more
FAILED: junit.framework.TestSuite.org.apache.solr.metrics.reporters.SolrJmxReporterCloudTest
Error Message:
Clean up static fields (in @AfterClass?), your test seems to hang on to approximately 21,905,360 bytes (threshold is 10,485,760). Field reference sizes (counted individually): - 21,904,744 bytes, private static javax.management.MBeanServer org.apache.solr.metrics.reporters.SolrJmxReporterCloudTest.mBeanServer - 192 bytes, public static org.junit.rules.TestRule org.apache.solr.SolrTestCaseJ4.solrClassRules - 128 bytes, private static java.lang.String org.apache.solr.SolrTestCaseJ4.factoryProp - 112 bytes, private static java.lang.String org.apache.solr.metrics.reporters.SolrJmxReporterCloudTest.COLLECTION - 72 bytes, private static java.util.Map org.apache.solr.SolrTestCaseJ4.savedClassLogLevels - 64 bytes, private static java.lang.String org.apache.solr.SolrTestCaseJ4.coreName - 48 bytes, private static java.lang.String org.apache.solr.SolrTestCaseJ4.initialRootLogLevel
Stack Trace:
junit.framework.AssertionFailedError: Clean up static fields (in @AfterClass?), your test seems to hang on to approximately 21,905,360 bytes (threshold is 10,485,760). Field reference sizes (counted individually):
- 21,904,744 bytes, private static javax.management.MBeanServer org.apache.solr.metrics.reporters.SolrJmxReporterCloudTest.mBeanServer
- 192 bytes, public static org.junit.rules.TestRule org.apache.solr.SolrTestCaseJ4.solrClassRules
- 128 bytes, private static java.lang.String org.apache.solr.SolrTestCaseJ4.factoryProp
- 112 bytes, private static java.lang.String org.apache.solr.metrics.reporters.SolrJmxReporterCloudTest.COLLECTION
- 72 bytes, private static java.util.Map org.apache.solr.SolrTestCaseJ4.savedClassLogLevels
- 64 bytes, private static java.lang.String org.apache.solr.SolrTestCaseJ4.coreName
- 48 bytes, private static java.lang.String org.apache.solr.SolrTestCaseJ4.initialRootLogLevel
at __randomizedtesting.SeedInfo.seed([4D57C60A96F4C56B]:0)
at com.carrotsearch.randomizedtesting.rules.StaticFieldsInvariantRule$1.afterAlways(StaticFieldsInvariantRule.java:170)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.security.TestPKIAuthenticationPlugin.test
Error Message:
Stack Trace:
java.lang.NullPointerException
at __randomizedtesting.SeedInfo.seed([4D57C60A96F4C56B:C503F9D03808A893]:0)
at org.apache.solr.security.TestPKIAuthenticationPlugin.test(TestPKIAuthenticationPlugin.java:100)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test
Error Message:
Captured an uncaught exception in thread: Thread[id=13695, name=Thread-3858, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=13695, name=Thread-3858, state=RUNNABLE, group=TGRP-FullSolrCloudDistribCmdsTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:39689/ij_b/collection2_shard2_replica_n4: ClusterState says we are the leader (http://127.0.0.1:39689/ij_b/collection2_shard2_replica_n4), but locally we don't think so. Request came from null
at __randomizedtesting.SeedInfo.seed([4D57C60A96F4C56B]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:627)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:549)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:993)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:862)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:793)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195)
at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest$1IndexThread.run(FullSolrCloudDistribCmdsTest.java:633)
FAILED: org.apache.solr.cloud.ShardSplitTest.testSplitAfterFailedSplit
Error Message:
expected:<1> but was:<2>
Stack Trace:
java.lang.AssertionError: expected:<1> but was:<2>
at __randomizedtesting.SeedInfo.seed([4D57C60A96F4C56B:B41A55A5AA8188E1]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.ShardSplitTest.testSplitAfterFailedSplit(ShardSplitTest.java:279)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.RollingRestartTest.test
Error Message:
Could not load collection from ZK: collection1
Stack Trace:
org.apache.solr.common.SolrException: Could not load collection from ZK: collection1
at __randomizedtesting.SeedInfo.seed([4D57C60A96F4C56B:C503F9D03808A893]:0)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1115)
at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:648)
at org.apache.solr.common.cloud.ClusterState.getCollectionOrNull(ClusterState.java:128)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.getTotalReplicas(AbstractFullDistribZkTestBase.java:488)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createJettys(AbstractFullDistribZkTestBase.java:441)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:334)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:991)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/state.json
at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1212)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:357)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:354)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:354)
at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1127)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1113)
... 38 more
FAILED: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test
Error Message:
expected:<2> but was:<4>
Stack Trace:
java.lang.AssertionError: expected:<2> but was:<4>
at __randomizedtesting.SeedInfo.seed([4D57C60A96F4C56B:C503F9D03808A893]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.assertSliceAndReplicaCount(SharedFSAutoReplicaFailoverTest.java:403)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.testBasics(SharedFSAutoReplicaFailoverTest.java:248)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test(SharedFSAutoReplicaFailoverTest.java:136)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 3598 lines...]
[junit4] Suite: org.apache.lucene.analysis.core.TestRandomChains
[junit4] 2> TEST FAIL: useCharFilter=false text=' \ubc25 \ud834\udc6e\ud834\udc76\ud834\udc3e \u467e\ued9e\u0003#\ufb63\u032d \u2c61\u2c62\u2c61\u2c76\u2c60\u2c7f \u30da\u30dd\u30f2\u30cf\u30d4 \u2182\u2150\u218f\u215a\u2166\u216b\u2157\u215b\u216a\u2172\u215c\u2165 udb \ua835\ua830\ua833\ua833\ua83b\ua836\ua834\ua83b\ua83a \udb51\udff0\ud85a\udfde'
[junit4] 2> Exception from random analyzer:
[junit4] 2> charfilters=
[junit4] 2> org.apache.lucene.analysis.pattern.PatternReplaceCharFilter(a, <KATAKANA>, java.io.StringReader@6a8a3a98)
[junit4] 2> tokenizer=
[junit4] 2> org.apache.lucene.analysis.standard.StandardTokenizer()
[junit4] 2> filters=
[junit4] 2> org.apache.lucene.analysis.shingle.ShingleFilter(ValidatingTokenFilter@3d7cdb0 term=,bytes=[],startOffset=0,endOffset=0,positionIncrement=1,positionLength=1,type=word,termFrequency=1)
[junit4] 2> org.apache.lucene.analysis.miscellaneous.PatternKeywordMarkerFilter(ValidatingTokenFilter@7001ffb8 term=,bytes=[],startOffset=0,endOffset=0,positionIncrement=1,positionLength=1,type=word,termFrequency=1,keyword=false, a)
[junit4] 2> org.apache.lucene.analysis.cjk.CJKBigramFilter(ValidatingTokenFilter@7398e9e7 term=,bytes=[],startOffset=0,endOffset=0,positionIncrement=1,positionLength=1,type=word,termFrequency=1,keyword=false, -43)
[junit4] 2> org.apache.lucene.analysis.miscellaneous.ASCIIFoldingFilter(ValidatingTokenFilter@6503b497 term=,bytes=[],startOffset=0,endOffset=0,positionIncrement=1,positionLength=1,type=word,termFrequency=1,keyword=false)
[junit4] 2> offsetsAreCorrect=false
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestRandomChains -Dtests.method=testRandomChainsWithLargeStrings -Dtests.seed=5700948A30BC1917 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=it-CH -Dtests.timezone=Africa/Luanda -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 8.89s J0 | TestRandomChains.testRandomChainsWithLargeStrings <<<
[junit4] > Throwable #1: java.lang.IllegalArgumentException: startOffset must be non-negative, and endOffset must be >= startOffset, and offsets must not go backwards startOffset=24,endOffset=31,lastStartOffset=27 for field 'dummy'
[junit4] > at __randomizedtesting.SeedInfo.seed([5700948A30BC1917:3D5B2B9B69F239E4]:0)
[junit4] > at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:767)
[junit4] > at org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:430)
[junit4] > at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:392)
[junit4] > at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:239)
[junit4] > at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:481)
[junit4] > at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1717)
[junit4] > at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1462)
[junit4] > at org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:171)
[junit4] > at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:650)
[junit4] > at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:540)
[junit4] > at org.apache.lucene.analysis.core.TestRandomChains.testRandomChainsWithLargeStrings(TestRandomChains.java:883)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/build/analysis/common/test/J0/temp/lucene.analysis.core.TestRandomChains_5700948A30BC1917-001
[junit4] 2> NOTE: test params are: codec=Lucene70, sim=RandomSimilarity(queryNorm=false): {dummy=DFR I(n)L1}, locale=it-CH, timezone=Africa/Luanda
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=185203120,total=275775488
[junit4] 2> NOTE: All tests run in this JVM: [TestBasqueAnalyzer, TestClassicAnalyzer, TestReverseStringFilter, TestSpanishAnalyzer, TestGreekStemmer, TestElision, TestElisionFilterFactory, TestTruncateTokenFilterFactory, WikipediaTokenizerTest, TestGalicianMinimalStemFilter, TestCustomAnalyzer, TestApostropheFilterFactory, TestCondition, TestPortugueseLightStemFilter, TestConcatenatingTokenStream, TestBengaliAnalyzer, TestStopAnalyzer, TestHunspellStemFilterFactory, TestThaiAnalyzer, TestDependencies, TestHungarianLightStemFilterFactory, TestHindiNormalizer, TestGermanMinimalStemFilter, TestIrishLowerCaseFilterFactory, TestTurkishAnalyzer, TestFullStrip, TestFixBrokenOffsetsFilter, TestCollationKeyAnalyzer, DateRecognizerFilterFactoryTest, TestKeywordMarkerFilterFactory, TestNorwegianAnalyzer, TestEscaped, TestEnglishAnalyzer, TestCommonGramsFilterFactory, TestNorwegianLightStemFilterFactory, TestGalicianMinimalStemFilterFactory, TestBulgarianStemFilterFactory, TestSoraniNormalizationFilter, TestLatvianAnalyzer, TestPorterStemFilter, TestPatternReplaceCharFilterFactory, TestFlattenGraphFilter, TestGermanAnalyzer, TestLimitTokenCountFilterFactory, TestKeywordMarkerFilter, TestFinnishAnalyzer, TestFactories, TestSnowball, TestRandomChains]
[junit4] Completed [107/284 (1!)] on J0 in 325.02s, 2 tests, 1 error <<< FAILURES!
[...truncated 5528 lines...]
[junit4] Suite: org.apache.lucene.search.TestInetAddressRangeQueries
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestInetAddressRangeQueries -Dtests.method=testRandomBig -Dtests.seed=FDEDA1026D94D0FC -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=sr-Latn-BA -Dtests.timezone=Europe/Chisinau -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] ERROR 761s J1 | TestInetAddressRangeQueries.testRandomBig <<<
[junit4] > Throwable #1: java.lang.OutOfMemoryError: Java heap space
[junit4] > at __randomizedtesting.SeedInfo.seed([FDEDA1026D94D0FC:7ABADC8DFCCDAC7C]:0)
[junit4] > at org.apache.lucene.util.bkd.HeapPointWriter.writePackedValue(HeapPointWriter.java:107)
[junit4] > at org.apache.lucene.util.bkd.HeapPointWriter.append(HeapPointWriter.java:128)
[junit4] > at org.apache.lucene.util.bkd.PointReader.split(PointReader.java:68)
[junit4] > at org.apache.lucene.util.bkd.BKDWriter.build(BKDWriter.java:1791)
[junit4] > at org.apache.lucene.util.bkd.BKDWriter.finish(BKDWriter.java:1008)
[junit4] > at org.apache.lucene.codecs.lucene60.Lucene60PointsWriter.writeField(Lucene60PointsWriter.java:130)
[junit4] > at org.apache.lucene.codecs.PointsWriter.mergeOneField(PointsWriter.java:62)
[junit4] > at org.apache.lucene.codecs.lucene60.Lucene60PointsWriter.merge(Lucene60PointsWriter.java:223)
[junit4] > at org.apache.lucene.index.SegmentMerger.mergePoints(SegmentMerger.java:187)
[junit4] > at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:136)
[junit4] > at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4392)
[junit4] > at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4032)
[junit4] > at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
[junit4] > at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2235)
[junit4] > at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5050)
[junit4] > at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1720)
[junit4] > at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1462)
[junit4] > at org.apache.lucene.search.BaseRangeFieldQueryTestCase.verify(BaseRangeFieldQueryTestCase.java:190)
[junit4] > at org.apache.lucene.search.BaseRangeFieldQueryTestCase.doTestRandom(BaseRangeFieldQueryTestCase.java:160)
[junit4] > at org.apache.lucene.search.BaseRangeFieldQueryTestCase.testRandomBig(BaseRangeFieldQueryTestCase.java:75)
[junit4] > at org.apache.lucene.search.TestInetAddressRangeQueries.testRandomBig(TestInetAddressRangeQueries.java:81)
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/build/misc/test/J1/temp/lucene.search.TestInetAddressRangeQueries_FDEDA1026D94D0FC-001
[junit4] 2> NOTE: test params are: codec=FastCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST, chunkSize=5, maxDocsPerChunk=2, blockSize=1), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST, chunkSize=5, blockSize=1)), sim=RandomSimilarity(queryNorm=false): {}, locale=sr-Latn-BA, timezone=Europe/Chisinau
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=256074464,total=319815680
[junit4] 2> NOTE: All tests run in this JVM: [TestHighFreqTerms, TestInetAddressRangeQueries]
[junit4] Completed [13/13 (1!)] on J1 in 842.09s, 4 tests, 1 error <<< FAILURES!
[...truncated 1 lines...]
[junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/build/misc/test/temp/junit4-J1-20170922_034655_3541775641769336547281.sysout
[junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
[junit4] java.lang.OutOfMemoryError: Java heap space
[junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/heapdumps/java_pid1505.hprof ...
[junit4] Heap dump file created [581695754 bytes in 6.665 secs]
[junit4] <<< JVM J1: EOF ----
[...truncated 3590 lines...]
[junit4] Suite: org.apache.solr.security.TestPKIAuthenticationPlugin
[junit4] 2> 0 INFO (SUITE-TestPKIAuthenticationPlugin-seed#[4D57C60A96F4C56B]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.security.TestPKIAuthenticationPlugin_4D57C60A96F4C56B-001/init-core-data-001
[junit4] 2> 1686 INFO (SUITE-TestPKIAuthenticationPlugin-seed#[4D57C60A96F4C56B]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 2295 INFO (SUITE-TestPKIAuthenticationPlugin-seed#[4D57C60A96F4C56B]-worker) [ ] o.e.j.u.log Logging initialized @19877ms
[junit4] 2> 2562 INFO (SUITE-TestPKIAuthenticationPlugin-seed#[4D57C60A96F4C56B]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 6935 INFO (TEST-TestPKIAuthenticationPlugin.test-seed#[4D57C60A96F4C56B]) [ ] o.a.s.SolrTestCaseJ4 ###Starting test
[junit4] 2> 23575 ERROR (TEST-TestPKIAuthenticationPlugin.test-seed#[4D57C60A96F4C56B]) [ ] o.a.s.s.PKIAuthenticationPlugin Invalid key request timestamp: 1506062262822 , received timestamp: 1506062277740 , TTL: 10000
[junit4] 2> 23578 INFO (TEST-TestPKIAuthenticationPlugin.test-seed#[4D57C60A96F4C56B]) [ ] o.a.s.SolrTestCaseJ4 ###Ending test
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestPKIAuthenticationPlugin -Dtests.method=test -Dtests.seed=4D57C60A96F4C56B -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=ar-KW -Dtests.timezone=Pacific/Tongatapu -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 16.9s J2 | TestPKIAuthenticationPlugin.test <<<
[junit4] > Throwable #1: java.lang.NullPointerException
[junit4] > at __randomizedtesting.SeedInfo.seed([4D57C60A96F4C56B:C503F9D03808A893]:0)
[junit4] > at org.apache.solr.security.TestPKIAuthenticationPlugin.test(TestPKIAuthenticationPlugin.java:100)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.security.TestPKIAuthenticationPlugin_4D57C60A96F4C56B-001
[junit4] 2> NOTE: test params are: codec=FastCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST, chunkSize=9, maxDocsPerChunk=164, blockSize=10), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST, chunkSize=9, blockSize=10)), sim=RandomSimilarity(queryNorm=false): {}, locale=ar-KW, timezone=Pacific/Tongatapu
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=185389432,total=253231104
[junit4] 2> NOTE: All tests run in this JVM: [TestPKIAuthenticationPlugin]
[junit4] Completed [1/732 (1!)] on J2 in 35.11s, 1 test, 1 error <<< FAILURES!
[...truncated 224 lines...]
[junit4] Suite: org.apache.solr.cloud.ShardSplitTest
[junit4] 2> 662397 INFO (SUITE-ShardSplitTest-seed#[4D57C60A96F4C56B]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ShardSplitTest_4D57C60A96F4C56B-001/init-core-data-001
[junit4] 2> 662398 INFO (SUITE-ShardSplitTest-seed#[4D57C60A96F4C56B]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 662399 INFO (SUITE-ShardSplitTest-seed#[4D57C60A96F4C56B]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
[junit4] 2> 662399 INFO (SUITE-ShardSplitTest-seed#[4D57C60A96F4C56B]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /_/s
[junit4] 2> 662439 INFO (TEST-ShardSplitTest.test-seed#[4D57C60A96F4C56B]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 662439 INFO (Thread-388) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 662439 INFO (Thread-388) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 663157 ERROR (Thread-388) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 663218 INFO (TEST-ShardSplitTest.test-seed#[4D57C60A96F4C56B]) [ ] o.a.s.c.ZkTestServer start zk server on port:54933
[junit4] 2> 664258 INFO (TEST-ShardSplitTest.test-seed#[4D57C60A96F4C56B]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 664276 INFO (TEST-ShardSplitTest.test-seed#[4D57C60A96F4C56B]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 664277 INFO (TEST-ShardSplitTest.test-seed#[4D57C60A96F4C56B]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 664278 INFO (TEST-ShardSplitTest.test-seed#[4D57C60A96F4C56B]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 664279 INFO (TEST-ShardSplitTest.test-seed#[4D57C60A96F4C56B]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 664279 INFO (TEST-ShardSplitTest.test-seed#[4D57C60A96F4C56B]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 664296 INFO (TEST-ShardSplitTest.test-seed#[4D57C60A96F4C56B]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 664297 INFO (TEST-ShardSplitTest.test-seed#[4D57C60A96F4C56B]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 664303 INFO (TEST-ShardSplitTest.test-seed#[4D57C60A96F4C56B]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 664321 INFO (TEST-ShardSplitTest.test-seed#[4D57C60A96F4C56B]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 664321 INFO (TEST-ShardSplitTest.test-seed#[4D57C60A96F4C56B]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 664335 INFO (TEST-ShardSplitTest.test-seed#[4D57C60A96F4C56B]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use TLOG replicas unless explicitly asked otherwise
[junit4] 2> 664877 INFO (TEST-ShardSplitTest.test-seed#[4D57C60A96F4C56B]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 665213 INFO (TEST-ShardSplitTest.test-seed#[4D57C60A96F4C56B]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4b2ce21{/_/s,null,AVAILABLE}
[junit4] 2> 665214 INFO (TEST-ShardSplitTest.test-seed#[4D57C60A96F4C56B]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@73ff59c4{HTTP/1.1,[http/1.1]}{127.0.0.1:33386}
[junit4] 2> 665214 INFO (TEST-ShardSplitTest.test-seed#[4D57C60A96F4C56B]) [ ] o.e.j.s.Server Started @687877ms
[junit4] 2> 665214 INFO (TEST-ShardSplitTest.test-seed#[4D57C60A96F4C56B]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ShardSplitTest_4D57C60A96F4C56B-001/tempDir-001/control/data, replicaType=NRT, hostContext=/_/s, hostPort=33386, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ShardSplitTest_4D57C60A96F4C56B-001/control-001/cores}
[junit4] 2> 665214 ERROR (TEST-ShardSplitTest.test-seed#[4D57C60A96F4C56B]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 665214 INFO (TEST-ShardSplitTest.test-seed#[4D57C60A96F4C56B]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
[junit4] 2> 665214 INFO (TEST-ShardSplitTest.test-seed#[4D57C60A96F4C56B]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 665214 INFO (TEST-ShardSplitTest.test-seed#[4D57C60A96F4C56B]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 665214 INFO (TEST-ShardSplitTest.test-seed#[4D57C60A96F4C56B]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-09-22T06:48:44.427Z
[junit4] 2> 665778 INFO (TEST-ShardSplitTest.test-seed#[4D57C60A96F4C56B]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 665778 INFO (TEST-ShardSplitTest.test-seed#[4D57C60A96F4C56B]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ShardSplitTest_4D57C60A96F4C56B-001/control-001/solr.xml
[junit4] 2> 665798 INFO (TEST-ShardSplitTest.test-seed#[4D57C60A96F4C56B]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@216aaf90, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 665812 INFO (TEST-ShardSplitTest.test-seed#[4D57C60A96F4C56B]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54933/solr
[junit4] 2> 666911 INFO (TEST-ShardSplitTest.test-seed#[4D57C60A96F4C56B]) [n:127.0.0.1:33386__%2Fs ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 666911 INFO (TEST-ShardSplitTest.test-seed#[4D57C60A96F4C56B]) [n:127.0.0.1:33386__%2Fs ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:33386__%2Fs
[junit4] 2> 666912 INFO (TEST-ShardSplitTest.test-seed#[4D57C60A96F4C56B]) [n:127.0.0.1:33386__%2Fs ] o.a.s.c.Overseer Overseer (id=98701339678015492-127.0.0.1:33386__%2Fs-n_0000000000) starting
[junit4] 2> 667168 INFO (TEST-ShardSplitTest.test-seed#[4D57C60A96F4C56B]) [n:127.0.0.1:33386__%2Fs ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33386__%2Fs
[junit4] 2> 667198 INFO (zkCallback-219-thread-1-processing-n:127.0.0.1:33386__%2Fs) [n:127.0.0.1:33386__%2Fs ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 667762 INFO (TEST-ShardSplitTest.test-seed#[4D57C60A96F4C56B]) [n:127.0.0.1:33386__%2Fs ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@216aaf90
[junit4] 2> 667820 INFO (TEST-ShardSplitTest.test-seed#[4D57C60A96F4C56B]) [n:127.0.0.1:33386__%2Fs ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@216aaf90
[junit4] 2> 667820 INFO (TEST-ShardSplitTest.test-seed#[4D57C60A96F4C56B]) [n:127.0.0.1:33386__%2Fs ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@216aaf90
[junit4] 2> 667848 INFO (TEST-ShardSplitTest.test-seed#[4D57C60A96F4C56B]) [n:127.0.0.1:33386__%2Fs ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ShardSplitTest_4D57C60A96F4C56B-001/control-001/cores
[junit4] 2> 668018 INFO (TEST-ShardSplitTest.test-seed#[4D57C60A96F4C56B]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 668018 INFO (TEST-ShardSplitTest.test-seed#[4D57C60A96F4C56B]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:54933/solr ready
[junit4] 2> 668048 INFO (qtp1825059431-1233) [n:127.0.0.1:33386__%2Fs ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:33386__%252Fs&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 668058 INFO (OverseerThreadFactory-473-thread-1-processing-n:127.0.0.1:33386__%2Fs) [n:127.0.0.1:33386__%2Fs ] o.a.s.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 668209 INFO (qtp1825059431-1231) [n:127.0.0.1:33386__%2Fs ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 668210 INFO (qtp1825059431-1231) [n:127.0.0.1:33386__%2Fs ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 668333 INFO (zkCallback-219-thread-1-processing-n:127.0.0.1:33386__%2Fs) [n:127.0.0.1:33386__%2Fs ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 669358 INFO (qtp1825059431-1231) [n:127.0.0.1:33386__%2Fs c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 669480 INFO (qtp1825059431-1231) [n:127.0.0.1:33386__%2Fs c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
[junit4] 2> 670668 INFO (qtp1825059431-1231) [n:127.0.0.1:33386__%2Fs c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 670837 INFO (qtp1825059431-1231) [n:127.0.0.1:33386__%2Fs c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
[junit4] 2> 670837 INFO (qtp1825059431-1231) [n:127.0.0.1:33386__%2Fs c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@216aaf90
[junit4] 2> 670837 INFO (qtp1825059431-1231) [n:127.0.0.1:33386__%2Fs c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 670837 INFO (qtp1825059431-1231) [n:127.0.0.1:33386__%2Fs c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ShardSplitTest_4D57C60A96F4C56B-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ShardSplitTest_4D57C60A96F4C56B-001/control-001/cores/control_collection_shard1_replica_n1/data/]
[junit4] 2> 672625 INFO (qtp1825059431-1231) [n:127.0.0.1:33386__%2Fs c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=14, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 672661 WARN (qtp1825059431-1231) [n:127.0.0.1:33386__%2Fs c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 673104 INFO (qtp1825059431-1231) [n:127.0.0.1:33386__%2Fs c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 673104 INFO (qtp1825059431-1231) [n:127.0.0.1:33386__%2Fs c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 673105 INFO (qtp1825059431-1231) [n:127.0.0.1:33386__%2Fs c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 673105 INFO (qtp1825059431-1231) [n:127.0.0.1:33386__%2Fs c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 673106 INFO (qtp1825059431-1231) [n:127.0.0.1:33386__%2Fs c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=9, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7874779351055387]
[junit4] 2> 673244 INFO (qtp1825059431-1231) [n:127.0.0.1:33386__%2Fs c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@54939a2f[control_collection_shard1_replica_n1] main]
[junit4] 2> 673272 INFO (qtp1825059431-1231) [n:127.0.0.1:33386__%2Fs c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 673273 INFO (qtp1825059431-1231) [n:127.0.0.1:33386__%2Fs c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 673273 INFO (qtp1825059431-1231) [n:127.0.0.1:33386__%2Fs c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 673273 INFO (qtp1825059431-1231) [n:127.0.0.1:33386__%2Fs c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1579221445494439936
[junit4] 2> 673279 INFO (zkCallback-219-thread-1-processing-n:127.0.0.1:33386__%2Fs) [n:127.0.0.1:33386__%2Fs ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 673282 INFO (qtp1825059431-1231) [n:127.0.0.1:33386__%2Fs c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 673282 INFO (qtp1825059431-1231) [n:127.0.0.1:33386__%2Fs c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 673282 INFO (qtp1825059431-1231) [n:127.0.0.1:33386__%2Fs c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:33386/_/s/control_collection_shard1_replica_n1/
[junit4] 2> 673282 INFO (qtp1825059431-1231) [n:127.0.0.1:33386__%2Fs c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 673282 INFO (qtp1825059431-1231) [n:127.0.0.1:33386__%2Fs c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:33386/_/s/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 673282 INFO (qtp1825059431-1231) [n:127.0.0.1:33386__%2Fs c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 673306 INFO (qtp1825059431-1231) [n:127.0.0.1:33386__%2Fs c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:33386/_/s/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 673326 INFO (searcherExecutor-476-thread-1-processing-n:127.0.0.1:33386__%2Fs x:control_collection_shard1_replica_n1 s:shard1 c:control_collection) [n:127.0.0.1:33386__%2Fs c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@54939a2f[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 673418 INFO (zkCallback-219-thread-1-processing-n:127.0.0.1:33386__%2Fs) [n:127.0.0.1:33386__%2Fs ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 673468 INFO (qtp1825059431-1231) [n:127.0.0.1:33386__%2Fs c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 673469 INFO (qtp1825059431-1231) [n:127.0.0.1:33386__%2Fs c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=con
[...truncated too long message...]
eckout/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/byte-buddy-1.6.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/commons-math3-3.6.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/hadoop-common-2.7.4-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/hadoop-hdfs-2.7.4-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/hadoop-minikdc-2.7.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/htrace-core-3.2.0-incubating.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/jetty-sslengine-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/mockito-core-2.6.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/netty-all-4.0.36.Final.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/objenesis-2.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/build/analysis/icu/lucene-analyzers-icu-7.1.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/contrib/analysis-extras/lib/icu4j-59.1.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-junit.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-log4j.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-junit4.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jai.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-javamail.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-bsf.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-commons-net.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-antlr.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jsch.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-oro.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-commons-logging.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-netrexx.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-testutil.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jdepend.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-bcel.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-xalan2.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-resolver.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jmf.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-regexp.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-swing.jar:/usr/local/asfpackages/java/jdk1.8.0_144/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.5.3.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/temp/junit4-J0-20170922_063715_7265916769008233607392.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/temp/junit4-J0-20170922_063715_7261655770784127556332.suites -stdin
[junit4] ERROR: JVM J0 ended with an exception: Forked process returned with error code: 1. Very likely a JVM crash. See process stdout at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/temp/junit4-J0-20170922_063715_7265814766250760612789.sysout See process stderr at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/temp/junit4-J0-20170922_063715_7268959493966597719047.syserr
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1519)
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:126)
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:982)
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:979)
[junit4] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] at java.lang.Thread.run(Thread.java:748)
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/build.xml:834: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/build.xml:770: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build.xml:267: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/common-build.xml:549: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/common-build.xml:1494: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/common-build.xml:1021: At least one slave process threw an exception, first: Forked process returned with error code: 1. Very likely a JVM crash. See process stdout at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/temp/junit4-J0-20170922_063715_7265814766250760612789.sysout See process stderr at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/temp/junit4-J0-20170922_063715_7268959493966597719047.syserr
Total time: 969 minutes 14 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[Fast Archiver] No artifacts from Lucene-Solr-NightlyTests-7.x #47 to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 48 - Failure
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/48/
16 tests failed.
FAILED: org.apache.lucene.codecs.memory.TestDirectDocValuesFormat.testSparseBinaryFixedLengthVsStoredFields
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([2AFB07A23520AF03]:0)
FAILED: junit.framework.TestSuite.org.apache.lucene.codecs.memory.TestDirectDocValuesFormat
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([2AFB07A23520AF03]:0)
FAILED: org.apache.solr.cloud.BasicDistributedZk2Test.test
Error Message:
Could not load collection from ZK: collection1
Stack Trace:
org.apache.solr.common.SolrException: Could not load collection from ZK: collection1
at __randomizedtesting.SeedInfo.seed([701F0B6A6DF5431C:F84B34B0C3092EE4]:0)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1115)
at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:648)
at org.apache.solr.client.solrj.impl.CloudSolrClient.getDocCollection(CloudSolrClient.java:1227)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:871)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:793)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:484)
at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:463)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.commit(AbstractFullDistribZkTestBase.java:1583)
at org.apache.solr.cloud.BasicDistributedZk2Test.brindDownShardIndexSomeDocsAndRecover(BasicDistributedZk2Test.java:299)
at org.apache.solr.cloud.BasicDistributedZk2Test.test(BasicDistributedZk2Test.java:108)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/state.json
at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1212)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:357)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:354)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:354)
at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1127)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1113)
... 51 more
FAILED: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test
Error Message:
Error from server at http://127.0.0.1:57210/_/w/control_collection: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 404 </title> </head> <body> <h2>HTTP ERROR: 404</h2> <p>Problem accessing /_/w/control_collection/update. Reason: <pre> Can not find: /_/w/control_collection/update</pre></p> <hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.20.v20170531</a><hr/> </body> </html>
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:57210/_/w/control_collection: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 404 </title>
</head>
<body>
<h2>HTTP ERROR: 404</h2>
<p>Problem accessing /_/w/control_collection/update. Reason:
<pre> Can not find: /_/w/control_collection/update</pre></p>
<hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.20.v20170531</a><hr/>
</body>
</html>
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:591)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:484)
at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:463)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.commit(AbstractFullDistribZkTestBase.java:1582)
at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testIndexingBatchPerRequestWithHttpSolrClient(FullSolrCloudDistribCmdsTest.java:656)
at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test(FullSolrCloudDistribCmdsTest.java:152)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.ShardSplitTest.testSplitAfterFailedSplit
Error Message:
expected:<1> but was:<2>
Stack Trace:
java.lang.AssertionError: expected:<1> but was:<2>
at __randomizedtesting.SeedInfo.seed([701F0B6A6DF5431C:895298C551800E96]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.ShardSplitTest.testSplitAfterFailedSplit(ShardSplitTest.java:279)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.TestPullReplica.testCreateDelete {seed=[701F0B6A6DF5431C:6B0FFA269B05BB2C]}
Error Message:
Error from server at https://127.0.0.1:41748/solr: KeeperErrorCode = NoNode for /overseer/collection-queue-work/qnr-0000000002
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:41748/solr: KeeperErrorCode = NoNode for /overseer/collection-queue-work/qnr-0000000002
at __randomizedtesting.SeedInfo.seed([701F0B6A6DF5431C:6B0FFA269B05BB2C]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:627)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1121)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:862)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:793)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195)
at org.apache.solr.cloud.TestPullReplica.testCreateDelete(TestPullReplica.java:182)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.TestStressInPlaceUpdates.stressTest
Error Message:
Error from server at http://127.0.0.1:34388/collection1: non ok status: 500, message:Server Error
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:34388/collection1: non ok status: 500, message:Server Error
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:557)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
at org.apache.solr.client.solrj.SolrClient.getById(SolrClient.java:1173)
at org.apache.solr.client.solrj.SolrClient.getById(SolrClient.java:1090)
at org.apache.solr.client.solrj.SolrClient.getById(SolrClient.java:1073)
at org.apache.solr.cloud.TestStressInPlaceUpdates.stressTest(TestStressInPlaceUpdates.java:440)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestTlogReplica
Error Message:
ObjectTracker found 5 object(s) that were not released!!! [MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, RawDirectoryWrapper] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348) at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:92) at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:741) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:934) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:843) at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:998) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:934) at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:91) at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384) at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177) at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:745) at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:726) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:507) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:426) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.eclipse.jetty.server.Server.handle(Server.java:534) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:251) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108) at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:484) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:332) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:419) at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$12(ReplicationHandler.java:1183) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.core.SolrCore at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1019) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:843) at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:998) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:934) at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:91) at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384) at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177) at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:745) at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:726) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:507) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:426) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.eclipse.jetty.server.Server.handle(Server.java:534) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:251) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108) at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348) at org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:478) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:928) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:843) at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:998) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:934) at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:91) at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384) at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177) at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:745) at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:726) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:507) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:426) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.eclipse.jetty.server.Server.handle(Server.java:534) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:251) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108) at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.RawDirectoryWrapper at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348) at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:349) at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:709) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:934) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:843) at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:998) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:934) at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:91) at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384) at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177) at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:745) at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:726) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:507) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:426) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.eclipse.jetty.server.Server.handle(Server.java:534) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:251) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108) at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
java.lang.AssertionError: ObjectTracker found 5 object(s) that were not released!!! [MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, RawDirectoryWrapper]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:92)
at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:741)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:934)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:843)
at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:998)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:934)
at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:91)
at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384)
at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389)
at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:745)
at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:726)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:507)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:426)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:534)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:251)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:484)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:332)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:419)
at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$12(ReplicationHandler.java:1183)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.core.SolrCore
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1019)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:843)
at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:998)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:934)
at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:91)
at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384)
at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389)
at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:745)
at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:726)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:507)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:426)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:534)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:251)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
at org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:478)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:928)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:843)
at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:998)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:934)
at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:91)
at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384)
at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389)
at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:745)
at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:726)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:507)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:426)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:534)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:251)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.RawDirectoryWrapper
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:349)
at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:709)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:934)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:843)
at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:998)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:934)
at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:91)
at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384)
at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389)
at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:745)
at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:726)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:507)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:426)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:534)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:251)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([701F0B6A6DF5431C]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:294)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:897)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.update.processor.UpdateRequestProcessorFactoryTest.testUpdateDistribChainSkipping
Error Message:
Tests must be run with INFO level logging otherwise LogUpdateProcessor isn't used and can't be tested.
Stack Trace:
java.lang.AssertionError: Tests must be run with INFO level logging otherwise LogUpdateProcessor isn't used and can't be tested.
at __randomizedtesting.SeedInfo.seed([701F0B6A6DF5431C:1FBF5BC1B1E64D0]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.update.processor.UpdateRequestProcessorFactoryTest.testUpdateDistribChainSkipping(UpdateRequestProcessorFactoryTest.java:101)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.util.TestSolrCLIRunExample
Error Message:
ObjectTracker found 1 object(s) that were not released!!! [TransactionLog] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.update.TransactionLog at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.update.TransactionLog.<init>(TransactionLog.java:190) at org.apache.solr.update.UpdateLog.newTransactionLog(UpdateLog.java:446) at org.apache.solr.update.UpdateLog.ensureLog(UpdateLog.java:1259) at org.apache.solr.update.UpdateLog.add(UpdateLog.java:532) at org.apache.solr.update.UpdateLog.add(UpdateLog.java:517) at org.apache.solr.update.DirectUpdateHandler2.doNormalUpdate(DirectUpdateHandler2.java:347) at org.apache.solr.update.DirectUpdateHandler2.addDoc0(DirectUpdateHandler2.java:266) at org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:216) at org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:67) at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:55) at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd(DistributedUpdateProcessor.java:991) at org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:1207) at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:753) at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:55) at org.apache.solr.update.processor.AddSchemaFieldsUpdateProcessorFactory$AddSchemaFieldsUpdateProcessor.processAdd(AddSchemaFieldsUpdateProcessorFactory.java:474) at org.apache.solr.handler.loader.JavabinLoader$1.update(JavabinLoader.java:98) at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readOuterMostDocIterator(JavaBinUpdateRequestCodec.java:188) at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readIterator(JavaBinUpdateRequestCodec.java:144) at org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinCodec.java:311) at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:256) at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readNamedList(JavaBinUpdateRequestCodec.java:130) at org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinCodec.java:276) at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:256) at org.apache.solr.common.util.JavaBinCodec.unmarshal(JavaBinCodec.java:178) at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.unmarshal(JavaBinUpdateRequestCodec.java:195) at org.apache.solr.handler.loader.JavabinLoader.parseAndLoadDocs(JavabinLoader.java:108) at org.apache.solr.handler.loader.JavabinLoader.load(JavabinLoader.java:55) at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:97) at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:68) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177) at org.apache.solr.core.SolrCore.execute(SolrCore.java:2483) at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:720) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:526) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:426) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.eclipse.jetty.server.Server.handle(Server.java:534) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108) at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
java.lang.AssertionError: ObjectTracker found 1 object(s) that were not released!!! [TransactionLog]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.update.TransactionLog
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.update.TransactionLog.<init>(TransactionLog.java:190)
at org.apache.solr.update.UpdateLog.newTransactionLog(UpdateLog.java:446)
at org.apache.solr.update.UpdateLog.ensureLog(UpdateLog.java:1259)
at org.apache.solr.update.UpdateLog.add(UpdateLog.java:532)
at org.apache.solr.update.UpdateLog.add(UpdateLog.java:517)
at org.apache.solr.update.DirectUpdateHandler2.doNormalUpdate(DirectUpdateHandler2.java:347)
at org.apache.solr.update.DirectUpdateHandler2.addDoc0(DirectUpdateHandler2.java:266)
at org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:216)
at org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:67)
at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:55)
at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd(DistributedUpdateProcessor.java:991)
at org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:1207)
at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:753)
at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:55)
at org.apache.solr.update.processor.AddSchemaFieldsUpdateProcessorFactory$AddSchemaFieldsUpdateProcessor.processAdd(AddSchemaFieldsUpdateProcessorFactory.java:474)
at org.apache.solr.handler.loader.JavabinLoader$1.update(JavabinLoader.java:98)
at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readOuterMostDocIterator(JavaBinUpdateRequestCodec.java:188)
at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readIterator(JavaBinUpdateRequestCodec.java:144)
at org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinCodec.java:311)
at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:256)
at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readNamedList(JavaBinUpdateRequestCodec.java:130)
at org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinCodec.java:276)
at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:256)
at org.apache.solr.common.util.JavaBinCodec.unmarshal(JavaBinCodec.java:178)
at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.unmarshal(JavaBinUpdateRequestCodec.java:195)
at org.apache.solr.handler.loader.JavabinLoader.parseAndLoadDocs(JavabinLoader.java:108)
at org.apache.solr.handler.loader.JavabinLoader.load(JavabinLoader.java:55)
at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:97)
at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:68)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2483)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:720)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:526)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:426)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:534)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([701F0B6A6DF5431C]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:294)
at sun.reflect.GeneratedMethodAccessor63.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:897)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.RollingRestartTest.test
Error Message:
No overseer leader found after restart #1: 127.0.0.1:37283_
Stack Trace:
java.lang.AssertionError: No overseer leader found after restart #1: 127.0.0.1:37283_
at __randomizedtesting.SeedInfo.seed([701F0B6A6DF5431C:F84B34B0C3092EE4]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.RollingRestartTest.restartWithRolesTest(RollingRestartTest.java:111)
at org.apache.solr.cloud.RollingRestartTest.test(RollingRestartTest.java:52)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.TestStressLiveNodes.testStress
Error Message:
iter1101 17 != 7 expected:<[127.0.0.1:60703_solr, thrasher-T1101_0-0, thrasher-T1101_0-1, thrasher-T1101_0-2, thrasher-T1101_0-3, thrasher-T1101_1-0, thrasher-T1101_1-1, thrasher-T1101_1-2, thrasher-T1101_1-3, thrasher-T1101_2-0, thrasher-T1101_2-1, thrasher-T1101_2-2, thrasher-T1101_2-3, thrasher-T1101_3-0, thrasher-T1101_3-1, thrasher-T1101_3-2, thrasher-T1101_3-3]> but was:<[127.0.0.1:60703_solr, thrasher-T1101_0-0, thrasher-T1101_0-1, thrasher-T1101_0-2, thrasher-T1101_0-3, thrasher-T1101_1-0, thrasher-T1101_1-1]>
Stack Trace:
java.lang.AssertionError: iter1101 17 != 7 expected:<[127.0.0.1:60703_solr, thrasher-T1101_0-0, thrasher-T1101_0-1, thrasher-T1101_0-2, thrasher-T1101_0-3, thrasher-T1101_1-0, thrasher-T1101_1-1, thrasher-T1101_1-2, thrasher-T1101_1-3, thrasher-T1101_2-0, thrasher-T1101_2-1, thrasher-T1101_2-2, thrasher-T1101_2-3, thrasher-T1101_3-0, thrasher-T1101_3-1, thrasher-T1101_3-2, thrasher-T1101_3-3]> but was:<[127.0.0.1:60703_solr, thrasher-T1101_0-0, thrasher-T1101_0-1, thrasher-T1101_0-2, thrasher-T1101_0-3, thrasher-T1101_1-0, thrasher-T1101_1-1]>
at __randomizedtesting.SeedInfo.seed([701F0B6A6DF5431C:650476ABEE02BB67]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.apache.solr.cloud.TestStressLiveNodes.testStress(TestStressLiveNodes.java:203)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(Abstract
[...truncated too long message...]
eckout/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/byte-buddy-1.6.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/commons-math3-3.6.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/hadoop-common-2.7.4-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/hadoop-hdfs-2.7.4-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/hadoop-minikdc-2.7.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/htrace-core-3.2.0-incubating.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/jetty-sslengine-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/mockito-core-2.6.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/netty-all-4.0.36.Final.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/test-lib/objenesis-2.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/build/analysis/icu/lucene-analyzers-icu-7.1.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/contrib/analysis-extras/lib/icu4j-59.1.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-junit.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-log4j.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-junit4.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jai.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-javamail.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-bsf.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-commons-net.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-antlr.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jsch.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-oro.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-commons-logging.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-netrexx.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-testutil.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jdepend.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-bcel.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-xalan2.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-resolver.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jmf.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-regexp.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-swing.jar:/usr/local/asfpackages/java/jdk1.8.0_144/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.5.3.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/temp/junit4-J2-20170919_150059_6127417593037403339300.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/temp/junit4-J2-20170919_150059_6126845419135413391029.suites -stdin
[junit4] ERROR: JVM J2 ended with an exception: Forked process returned with error code: 1. Very likely a JVM crash. See process stdout at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/temp/junit4-J2-20170919_150059_6126790465962460662718.sysout See process stderr at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/temp/junit4-J2-20170919_150059_6125719714417966345181.syserr
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1519)
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:126)
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:982)
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:979)
[junit4] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] at java.lang.Thread.run(Thread.java:748)
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/build.xml:834: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/build.xml:770: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build.xml:267: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/common-build.xml:549: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/common-build.xml:1494: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/common-build.xml:1021: At least one slave process threw an exception, first: Forked process returned with error code: 1. Very likely a JVM crash. See process stdout at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/temp/junit4-J1-20170919_150059_6122540141637564670305.sysout See process stderr at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/temp/junit4-J1-20170919_150059_6124626163578896115691.syserr
Total time: 1131 minutes 3 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[Fast Archiver] No artifacts from Lucene-Solr-NightlyTests-7.x #47 to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 47 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/47/
12 tests failed.
FAILED: org.apache.lucene.index.TestIndexSorting.testRandom3
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([1A90254FFA0EC131]:0)
FAILED: junit.framework.TestSuite.org.apache.lucene.index.TestIndexSorting
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([1A90254FFA0EC131]:0)
FAILED: org.apache.solr.cloud.DistribJoinFromCollectionTest.testNoScore
Error Message:
Expected 1 doc, got {numFound=0,start=0,maxScore=0.0,docs=[]}
Stack Trace:
java.lang.AssertionError: Expected 1 doc, got {numFound=0,start=0,maxScore=0.0,docs=[]}
at __randomizedtesting.SeedInfo.seed([FFBA544AA6232C62:403558F8586F332E]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.DistribJoinFromCollectionTest.testJoins(DistribJoinFromCollectionTest.java:155)
at org.apache.solr.cloud.DistribJoinFromCollectionTest.testNoScore(DistribJoinFromCollectionTest.java:121)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test
Error Message:
Expected 2 of 3 replicas to be active but only found 1; [core_node4:{"core":"c8n_1x3_lf_shard1_replica_n1","base_url":"http://127.0.0.1:58973","node_name":"127.0.0.1:58973_","state":"active","type":"NRT","leader":"true"}]; clusterState: DocCollection(c8n_1x3_lf//clusterstate.json/30)={ "pullReplicas":"0", "replicationFactor":"1", "shards":{"shard1":{ "range":"80000000-7fffffff", "state":"active", "replicas":{ "core_node4":{ "core":"c8n_1x3_lf_shard1_replica_n1", "base_url":"http://127.0.0.1:58973", "node_name":"127.0.0.1:58973_", "state":"active", "type":"NRT", "leader":"true"}, "core_node5":{ "core":"c8n_1x3_lf_shard1_replica_n2", "base_url":"http://127.0.0.1:57590", "node_name":"127.0.0.1:57590_", "state":"down", "type":"NRT"}, "core_node6":{ "state":"down", "base_url":"http://127.0.0.1:55501", "core":"c8n_1x3_lf_shard1_replica_n3", "node_name":"127.0.0.1:55501_", "type":"NRT"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"false", "nrtReplicas":"3", "tlogReplicas":"0"}
Stack Trace:
java.lang.AssertionError: Expected 2 of 3 replicas to be active but only found 1; [core_node4:{"core":"c8n_1x3_lf_shard1_replica_n1","base_url":"http://127.0.0.1:58973","node_name":"127.0.0.1:58973_","state":"active","type":"NRT","leader":"true"}]; clusterState: DocCollection(c8n_1x3_lf//clusterstate.json/30)={
"pullReplicas":"0",
"replicationFactor":"1",
"shards":{"shard1":{
"range":"80000000-7fffffff",
"state":"active",
"replicas":{
"core_node4":{
"core":"c8n_1x3_lf_shard1_replica_n1",
"base_url":"http://127.0.0.1:58973",
"node_name":"127.0.0.1:58973_",
"state":"active",
"type":"NRT",
"leader":"true"},
"core_node5":{
"core":"c8n_1x3_lf_shard1_replica_n2",
"base_url":"http://127.0.0.1:57590",
"node_name":"127.0.0.1:57590_",
"state":"down",
"type":"NRT"},
"core_node6":{
"state":"down",
"base_url":"http://127.0.0.1:55501",
"core":"c8n_1x3_lf_shard1_replica_n3",
"node_name":"127.0.0.1:55501_",
"type":"NRT"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"3",
"tlogReplicas":"0"}
at __randomizedtesting.SeedInfo.seed([FFBA544AA6232C62:77EE6B9008DF419A]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.testRf3WithLeaderFailover(LeaderFailoverAfterPartitionTest.java:169)
at org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test(LeaderFailoverAfterPartitionTest.java:56)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.RollingRestartTest.test
Error Message:
KeeperErrorCode = Session expired for /clusterstate.json
Stack Trace:
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /clusterstate.json
at __randomizedtesting.SeedInfo.seed([FFBA544AA6232C62:77EE6B9008DF419A]:0)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1212)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:357)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:354)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:354)
at org.apache.solr.common.cloud.ZkStateReader.refreshLegacyClusterState(ZkStateReader.java:542)
at org.apache.solr.common.cloud.ZkStateReader.forceUpdateCollection(ZkStateReader.java:310)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.updateMappingsFromZk(AbstractFullDistribZkTestBase.java:674)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.updateMappingsFromZk(AbstractFullDistribZkTestBase.java:669)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createJettys(AbstractFullDistribZkTestBase.java:464)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:334)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:991)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsRestartWhileUpdatingTest
Error Message:
4 threads leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsRestartWhileUpdatingTest: 1) Thread[id=7288, name=searcherExecutor-2506-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=7258, name=searcherExecutor-2495-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=7222, name=searcherExecutor-2484-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=7336, name=searcherExecutor-2517-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 4 threads leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsRestartWhileUpdatingTest:
1) Thread[id=7288, name=searcherExecutor-2506-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=7258, name=searcherExecutor-2495-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=7222, name=searcherExecutor-2484-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
4) Thread[id=7336, name=searcherExecutor-2517-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([FFBA544AA6232C62]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsRestartWhileUpdatingTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=7288, name=searcherExecutor-2506-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=7258, name=searcherExecutor-2495-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=7222, name=searcherExecutor-2484-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=7336, name=searcherExecutor-2517-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=7288, name=searcherExecutor-2506-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=7258, name=searcherExecutor-2495-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=7222, name=searcherExecutor-2484-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
4) Thread[id=7336, name=searcherExecutor-2517-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([FFBA544AA6232C62]:0)
FAILED: org.apache.solr.common.cloud.TestCollectionStateWatchers.testWaitForStateWatcherIsRetainedOnPredicateFailure
Error Message:
Did not see a fully active cluster after 30 seconds
Stack Trace:
java.lang.AssertionError: Did not see a fully active cluster after 30 seconds
at __randomizedtesting.SeedInfo.seed([69B82F5134D44E35:E18E8D02EC7BA627]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.common.cloud.TestCollectionStateWatchers.testWaitForStateWatcherIsRetainedOnPredicateFailure(TestCollectionStateWatchers.java:250)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.hdfs.HdfsRestartWhileUpdatingTest.test
Error Message:
KeeperErrorCode = Session expired for /clusterstate.json
Stack Trace:
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /clusterstate.json
at __randomizedtesting.SeedInfo.seed([FFBA544AA6232C62:77EE6B9008DF419A]:0)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1212)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:357)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:354)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:354)
at org.apache.solr.common.cloud.ZkStateReader.refreshLegacyClusterState(ZkStateReader.java:542)
at org.apache.solr.common.cloud.ZkStateReader.forceUpdateCollection(ZkStateReader.java:310)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.updateMappingsFromZk(AbstractFullDistribZkTestBase.java:674)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.updateMappingsFromZk(AbstractFullDistribZkTestBase.java:669)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createJettys(AbstractFullDistribZkTestBase.java:464)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:334)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:991)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_dv
Error Message:
java.lang.RuntimeException: Error from server at http://127.0.0.1:39599/solr/test_col: Async exception during distributed update: Error from server at http://127.0.0.1:43329/solr/test_col_shard3_replica_n11: Server Error request: http://127.0.0.1:43329/solr/test_col_shard3_replica_n11/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A39599%2Fsolr%2Ftest_col_shard5_replica_n18%2F&wt=javabin&version=2 Remote error message: Failed synchronous update on shard StdNode: http://127.0.0.1:52956/solr/test_col_shard3_replica_n9/ update: org.apache.solr.client.solrj.request.UpdateRequest@2f2e71a
Stack Trace:
java.util.concurrent.ExecutionException: java.lang.RuntimeException: Error from server at http://127.0.0.1:39599/solr/test_col: Async exception during distributed update: Error from server at http://127.0.0.1:43329/solr/test_col_shard3_replica_n11: Server Error
request: http://127.0.0.1:43329/solr/test_col_shard3_replica_n11/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A39599%2Fsolr%2Ftest_col_shard5_replica_n18%2F&wt=javabin&version=2
Remote error message: Failed synchronous update on shard StdNode: http://127.0.0.1:52956/solr/test_col_shard3_replica_n9/ update: org.apache.solr.client.solrj.request.UpdateRequest@2f2e71a
at __randomizedtesting.SeedInfo.seed([FFBA544AA6232C62:C9AE360C2C7E1673]:0)
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.checkField(TestStressCloudBlindAtomicUpdates.java:283)
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_dv(TestStressCloudBlindAtomicUpdates.java:195)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: Error from server at http://127.0.0.1:39599/solr/test_col: Async exception during distributed update: Error from server at http://127.0.0.1:43329/solr/test_col_shard3_replica_n11: Server Error
request: http://127.0.0.1:43329/solr/test_col_shard3_replica_n11/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A39599%2Fsolr%2Ftest_col_shard5_replica_n18%2F&wt=javabin&version=2
Remote error message: Failed synchronous update on shard StdNode: http://127.0.0.1:52956/solr/test_col_shard3_replica_n9/ update: org.apache.solr.client.solrj.request.UpdateRequest@2f2e71a
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.run(TestStressCloudBlindAtomicUpdates.java:411)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:39599/solr/test_col: Async exception during distributed update: Error from server at http://127.0.0.1:43329/solr/test_col_shard3_replica_n11: Server Error
request: http://127.0.0.1:43329/solr/test_col_shard3_replica_n11/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A39599%2Fsolr%2Ftest_col_shard5_replica_n18%2F&wt=javabin&version=2
Remote error message: Failed synchronous update on shard StdNode: http://127.0.0.1:52956/solr/test_col_shard3_replica_n9/ update: org.apache.solr.client.solrj.request.UpdateRequest@2f2e71a
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:627)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195)
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.doRandomAtomicUpdate(TestStressCloudBlindAtomicUpdates.java:370)
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.run(TestStressCloudBlindAtomicUpdates.java:399)
... 6 more
FAILED: org.apache.solr.cloud.hdfs.HdfsUnloadDistributedZkTest.test
Error Message:
Captured an uncaught exception in thread: Thread[id=3261, name=testExecutor-1023-thread-12, state=RUNNABLE, group=TGRP-HdfsUnloadDistributedZkTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=3261, name=testExecutor-1023-thread-12, state=RUNNABLE, group=TGRP-HdfsUnloadDistributedZkTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:41134/d_lb/bx
at __randomizedtesting.SeedInfo.seed([FFBA544AA6232C62]:0)
at org.apache.solr.cloud.BasicDistributedZkTest.lambda$createCollectionInOneInstance$0(BasicDistributedZkTest.java:595)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:41134/d_lb/bx
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:638)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195)
at org.apache.solr.cloud.BasicDistributedZkTest.lambda$createCollectionInOneInstance$0(BasicDistributedZkTest.java:593)
... 4 more
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:526)
... 9 more
FAILED: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test
Error Message:
expected:<2> but was:<4>
Stack Trace:
java.lang.AssertionError: expected:<2> but was:<4>
at __randomizedtesting.SeedInfo.seed([FFBA544AA6232C62:77EE6B9008DF419A]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.assertSliceAndReplicaCount(SharedFSAutoReplicaFailoverTest.java:403)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.testBasics(SharedFSAutoReplicaFailoverTest.java:248)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test(SharedFSAutoReplicaFailoverTest.java:136)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 1739 lines...]
[junit4] Suite: org.apache.lucene.index.TestIndexSorting
[junit4] 2> Set 17, 2017 8:45:18 QN com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
[junit4] 2> WARNING: Suite execution timed out: org.apache.lucene.index.TestIndexSorting
[junit4] 2> 1) Thread[id=1, name=main, state=WAITING, group=main]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Thread.join(Thread.java:1252)
[junit4] 2> at java.lang.Thread.join(Thread.java:1326)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:636)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:493)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:251)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:368)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:13)
[junit4] 2> 2) Thread[id=1093, name=SUITE-TestIndexSorting-seed#[1A90254FFA0EC131], state=RUNNABLE, group=TGRP-TestIndexSorting]
[junit4] 2> at java.lang.Thread.getStackTrace(Thread.java:1559)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:696)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:693)
[junit4] 2> at java.security.AccessController.doPrivileged(Native Method)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getStackTrace(ThreadLeakControl.java:693)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getThreadsWithTraces(ThreadLeakControl.java:709)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:689)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:65)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:415)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:705)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:139)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:626)
[junit4] 2> 3) Thread[id=9, name=JUnit4-serializer-daemon, state=TIMED_WAITING, group=main]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:50)
[junit4] 2> 4) Thread[id=1094, name=TEST-TestIndexSorting.testRandom3-seed#[1A90254FFA0EC131], state=RUNNABLE, group=TGRP-TestIndexSorting]
[junit4] 2> at org.apache.lucene.codecs.compressing.LZ4.decompress(LZ4.java:135)
[junit4] 2> at org.apache.lucene.codecs.compressing.CompressionMode$4.decompress(CompressionMode.java:138)
[junit4] 2> at org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader$BlockState.document(CompressingStoredFieldsReader.java:555)
[junit4] 2> at org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader.document(CompressingStoredFieldsReader.java:571)
[junit4] 2> at org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader.visitDocument(CompressingStoredFieldsReader.java:578)
[junit4] 2> at org.apache.lucene.index.CodecReader.document(CodecReader.java:83)
[junit4] 2> at org.apache.lucene.index.BaseCompositeReader.document(BaseCompositeReader.java:118)
[junit4] 2> at org.apache.lucene.index.IndexReader.document(IndexReader.java:360)
[junit4] 2> at org.apache.lucene.index.TestIndexSorting.testRandom3(TestIndexSorting.java:2364)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestIndexSorting -Dtests.method=testRandom3 -Dtests.seed=1A90254FFA0EC131 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=mt-MT -Dtests.timezone=America/Scoresbysund -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 7053s J1 | TestIndexSorting.testRandom3 <<<
[junit4] > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
[junit4] > at __randomizedtesting.SeedInfo.seed([1A90254FFA0EC131]:0)
[junit4] 2> Set 17, 2017 8:45:18 QN com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> Set 17, 2017 8:45:38 QN com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.lucene.index.TestIndexSorting:
[junit4] 2> 1) Thread[id=1094, name=TEST-TestIndexSorting.testRandom3-seed#[1A90254FFA0EC131], state=RUNNABLE, group=TGRP-TestIndexSorting]
[junit4] 2> at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:159)
[junit4] 2> at org.apache.lucene.mockfile.FilterSeekableByteChannel.read(FilterSeekableByteChannel.java:59)
[junit4] 2> at org.apache.lucene.mockfile.FilterSeekableByteChannel.read(FilterSeekableByteChannel.java:59)
[junit4] 2> at org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput.readInternal(SimpleFSDirectory.java:178)
[junit4] 2> at org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:342)
[junit4] 2> at org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:54)
[junit4] 2> at org.apache.lucene.store.DataInput.readVInt(DataInput.java:125)
[junit4] 2> at org.apache.lucene.store.BufferedIndexInput.readVInt(BufferedIndexInput.java:221)
[junit4] 2> at org.apache.lucene.store.MockIndexInputWrapper.readVInt(MockIndexInputWrapper.java:183)
[junit4] 2> at org.apache.lucene.codecs.lucene70.Lucene70DocValuesProducer$TermsDict.getFirstTermFromBlock(Lucene70DocValuesProducer.java:1080)
[junit4] 2> at org.apache.lucene.codecs.lucene70.Lucene70DocValuesProducer$TermsDict.seekBlock(Lucene70DocValuesProducer.java:1099)
[junit4] 2> at org.apache.lucene.codecs.lucene70.Lucene70DocValuesProducer$TermsDict.seekCeil(Lucene70DocValuesProducer.java:1116)
[junit4] 2> at org.apache.lucene.codecs.lucene70.Lucene70DocValuesProducer$BaseSortedDocValues.lookupTerm(Lucene70DocValuesProducer.java:931)
[junit4] 2> at org.apache.lucene.search.FieldComparator$TermOrdValComparator.setBottom(FieldComparator.java:757)
[junit4] 2> at org.apache.lucene.search.MultiLeafFieldComparator.setBottom(MultiLeafFieldComparator.java:44)
[junit4] 2> at org.apache.lucene.search.TopFieldCollector$SimpleFieldCollector$1.collect(TopFieldCollector.java:131)
[junit4] 2> at org.apache.lucene.search.FilterLeafCollector.collect(FilterLeafCollector.java:43)
[junit4] 2> at org.apache.lucene.search.EarlyTerminatingSortingCollector$1.collect(EarlyTerminatingSortingCollector.java:113)
[junit4] 2> at org.apache.lucene.search.AssertingLeafCollector.collect(AssertingLeafCollector.java:52)
[junit4] 2> at org.apache.lucene.search.AssertingCollector$1.collect(AssertingCollector.java:56)
[junit4] 2> at org.apache.lucene.search.AssertingLeafCollector.collect(AssertingLeafCollector.java:52)
[junit4] 2> at org.apache.lucene.search.AssertingLeafCollector.collect(AssertingLeafCollector.java:52)
[junit4] 2> at org.apache.lucene.search.MatchAllDocsQuery$1$1.score(MatchAllDocsQuery.java:56)
[junit4] 2> at org.apache.lucene.search.AssertingBulkScorer.score(AssertingBulkScorer.java:79)
[junit4] 2> at org.apache.lucene.search.AssertingBulkScorer.score(AssertingBulkScorer.java:63)
[junit4] 2> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:658)
[junit4] 2> at org.apache.lucene.search.AssertingIndexSearcher.search(AssertingIndexSearcher.java:72)
[junit4] 2> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:462)
[junit4] 2> at org.apache.lucene.index.TestIndexSorting.testRandom3(TestIndexSorting.java:2344)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Set 17, 2017 8:45:38 QN com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=1094, name=TEST-TestIndexSorting.testRandom3-seed#[1A90254FFA0EC131], state=RUNNABLE, group=TGRP-TestIndexSorting]
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/build/core/test/J1/temp/lucene.index.TestIndexSorting_1A90254FFA0EC131-001
[junit4] 2> Sep 17, 2017 8:45:38 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: All leaked threads terminated.
[junit4] 2> NOTE: test params are: codec=FastCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST, chunkSize=24812, maxDocsPerChunk=300, blockSize=546), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST, chunkSize=24812, blockSize=546)), sim=RandomSimilarity(queryNorm=true): {positions=DFR G1, id=DFR I(F)LZ(0.3), term_vectors=DFR I(F)2}, locale=mt-MT, timezone=America/Scoresbysund
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=295698448,total=477626368
[junit4] 2> NOTE: All tests run in this JVM: [TestAtomicUpdate, TestSpanExplanationsOfNonMatches, TestIndexWriterFromReader, TestMultiTermConstantScore, TestPointValues, TestAddIndexes, TestTermVectorsReader, TestFixedBitDocIdSet, TestQueryBuilder, TestFilterSpans, TestNorms, Test2BPostings, TestLRUQueryCache, TestMixedDocValuesUpdates, TestIndexSorting]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestIndexSorting -Dtests.seed=1A90254FFA0EC131 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=mt-MT -Dtests.timezone=America/Scoresbysund -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J1 | TestIndexSorting (suite) <<<
[junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4] > at __randomizedtesting.SeedInfo.seed([1A90254FFA0EC131]:0)
[junit4] Completed [328/454 (1!)] on J1 in 7220.69s, 36 tests, 2 errors <<< FAILURES!
[...truncated 11250 lines...]
[junit4] Suite: org.apache.solr.cloud.hdfs.HdfsUnloadDistributedZkTest
[junit4] 2> 1654343 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[FFBA544AA6232C62]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_FFBA544AA6232C62-001/init-core-data-001
[junit4] 2> 1654343 WARN (SUITE-HdfsUnloadDistributedZkTest-seed#[FFBA544AA6232C62]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=16 numCloses=16
[junit4] 2> 1654343 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[FFBA544AA6232C62]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 1654362 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[FFBA544AA6232C62]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
[junit4] 2> 1654362 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[FFBA544AA6232C62]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /d_lb/bx
[junit4] 2> 1659982 WARN (SUITE-HdfsUnloadDistributedZkTest-seed#[FFBA544AA6232C62]-worker) [ ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 1669093 WARN (SUITE-HdfsUnloadDistributedZkTest-seed#[FFBA544AA6232C62]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 1672088 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[FFBA544AA6232C62]-worker) [ ] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
[junit4] 2> 1672317 WARN (SUITE-HdfsUnloadDistributedZkTest-seed#[FFBA544AA6232C62]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 1673346 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[FFBA544AA6232C62]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 1673583 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[FFBA544AA6232C62]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_52100_hdfs____e949um/webapp
[junit4] 2> 1678024 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[FFBA544AA6232C62]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:52100
[junit4] 2> 1693146 WARN (SUITE-HdfsUnloadDistributedZkTest-seed#[FFBA544AA6232C62]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 1693200 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[FFBA544AA6232C62]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 1693387 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[FFBA544AA6232C62]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_52189_datanode____41dcpo/webapp
[junit4] 2> 1696731 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[FFBA544AA6232C62]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:52189
[junit4] 2> 1704138 WARN (SUITE-HdfsUnloadDistributedZkTest-seed#[FFBA544AA6232C62]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 1704157 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[FFBA544AA6232C62]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 1704507 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[FFBA544AA6232C62]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_42606_datanode____.pbvnob/webapp
[junit4] 2> 1707725 INFO (SUITE-HdfsUnloadDistributedZkTest-seed#[FFBA544AA6232C62]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42606
[junit4] 2> 1716732 ERROR (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_FFBA544AA6232C62-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_FFBA544AA6232C62-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to localhost/127.0.0.1:59555) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 1716742 ERROR (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_FFBA544AA6232C62-001/tempDir-001/hdfsBaseDir/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_FFBA544AA6232C62-001/tempDir-001/hdfsBaseDir/data/data4/]] heartbeating to localhost/127.0.0.1:59555) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 1720250 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x15090345590a61: from storage DS-d4c53f0f-37d8-4a72-913c-cde82be663d3 node DatanodeRegistration(127.0.0.1:60258, datanodeUuid=550a4ed9-30f0-4f47-8ee6-2da39755465f, infoPort=57889, infoSecurePort=0, ipcPort=50728, storageInfo=lv=-56;cid=testClusterID;nsid=366817470;c=0), blocks: 0, hasStaleStorage: true, processing time: 9 msecs
[junit4] 2> 1720252 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x15090345590a61: from storage DS-07bd685d-ac6b-47fe-88be-548fc57d459c node DatanodeRegistration(127.0.0.1:60258, datanodeUuid=550a4ed9-30f0-4f47-8ee6-2da39755465f, infoPort=57889, infoSecurePort=0, ipcPort=50728, storageInfo=lv=-56;cid=testClusterID;nsid=366817470;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 1720315 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x15090346c96b5d: from storage DS-f5186924-7fdb-4dac-b40d-beca69c835bf node DatanodeRegistration(127.0.0.1:57658, datanodeUuid=a006ec4a-fac5-4dd5-a1e3-4f7413398546, infoPort=47399, infoSecurePort=0, ipcPort=55343, storageInfo=lv=-56;cid=testClusterID;nsid=366817470;c=0), blocks: 0, hasStaleStorage: true, processing time: 1 msecs
[junit4] 2> 1720316 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x15090346c96b5d: from storage DS-0687868e-be76-48ad-bad0-130e7f98e18a node DatanodeRegistration(127.0.0.1:57658, datanodeUuid=a006ec4a-fac5-4dd5-a1e3-4f7413398546, infoPort=47399, infoSecurePort=0, ipcPort=55343, storageInfo=lv=-56;cid=testClusterID;nsid=366817470;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 1724067 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[FFBA544AA6232C62]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1724073 INFO (Thread-590) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1724073 INFO (Thread-590) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1724098 ERROR (Thread-590) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1724207 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[FFBA544AA6232C62]) [ ] o.a.s.c.ZkTestServer start zk server on port:39385
[junit4] 2> 1725021 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[FFBA544AA6232C62]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 1725030 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[FFBA544AA6232C62]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 1725044 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[FFBA544AA6232C62]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1725045 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[FFBA544AA6232C62]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 1725046 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[FFBA544AA6232C62]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 1725054 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[FFBA544AA6232C62]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 1725055 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[FFBA544AA6232C62]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 1725056 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[FFBA544AA6232C62]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 1725057 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[FFBA544AA6232C62]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1725057 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[FFBA544AA6232C62]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 1725061 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[FFBA544AA6232C62]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 1725073 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[FFBA544AA6232C62]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
[junit4] 2> 1726627 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[FFBA544AA6232C62]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 1726647 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[FFBA544AA6232C62]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@271677c3{/d_lb/bx,null,AVAILABLE}
[junit4] 2> 1726647 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[FFBA544AA6232C62]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1ae6d209{HTTP/1.1,[http/1.1]}{127.0.0.1:46993}
[junit4] 2> 1726647 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[FFBA544AA6232C62]) [ ] o.e.j.s.Server Started @1747546ms
[junit4] 2> 1726647 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[FFBA544AA6232C62]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:59555/hdfs__localhost_59555__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-7.x_checkout_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.HdfsUnloadDistributedZkTest_FFBA544AA6232C62-001_tempDir-002_control_data, replicaType=NRT, hostContext=/d_lb/bx, hostPort=46993, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_FFBA544AA6232C62-001/control-001/cores}
[junit4] 2> 1726647 ERROR (TEST-HdfsUnloadDistributedZkTest.test-seed#[FFBA544AA6232C62]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1726647 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[FFBA544AA6232C62]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
[junit4] 2> 1726648 INFO (TEST-HdfsUnloadDistributedZkTest.test-seed#[FFBA544AA6232C62]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2>
[...truncated too long message...]
edEvent state:SyncConnected type:NodeDataChanged path:/collections/falsepredicate/state.json] for collection [falsepredicate] has occurred - updating... (live nodes size: [3])
[junit4] 2> 470039 INFO (zkCallback-104-thread-1-processing-n:127.0.0.1:55978_solr) [n:127.0.0.1:55978_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/falsepredicate/state.json] for collection [falsepredicate] has occurred - updating... (live nodes size: [3])
[junit4] 2> 470054 INFO (zkCallback-103-thread-6-processing-n:127.0.0.1:40241_solr) [n:127.0.0.1:40241_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/falsepredicate/state.json] for collection [falsepredicate] has occurred - updating... (live nodes size: [3])
[junit4] 2> 470054 INFO (zkCallback-103-thread-1-processing-n:127.0.0.1:40241_solr) [n:127.0.0.1:40241_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/falsepredicate/state.json] for collection [falsepredicate] has occurred - updating... (live nodes size: [3])
[junit4] 2> 470055 INFO (zkCallback-121-thread-1-processing-n:127.0.0.1:45706_solr) [n:127.0.0.1:45706_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/currentstate/state.json] for collection [currentstate] has occurred - updating... (live nodes size: [3])
[junit4] 2> 470055 INFO (zkCallback-121-thread-3-processing-n:127.0.0.1:45706_solr) [n:127.0.0.1:45706_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/currentstate/state.json] for collection [currentstate] has occurred - updating... (live nodes size: [3])
[junit4] 2> 470060 INFO (jetty-closer-83-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@7074d81f{/solr,null,UNAVAILABLE}
[junit4] 2> 470075 INFO (zkCallback-121-thread-3-processing-n:127.0.0.1:45706_solr) [n:127.0.0.1:45706_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 470091 INFO (zkCallback-103-thread-6-processing-n:127.0.0.1:40241_solr) [n:127.0.0.1:40241_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 470111 INFO (coreCloseExecutor-374-thread-2) [n:127.0.0.1:45706_solr c:currentstate s:shard1 r:core_node2 x:currentstate_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.currentstate.shard1.replica_n1, tag=975361776
[junit4] 2> 470111 INFO (coreCloseExecutor-374-thread-2) [n:127.0.0.1:45706_solr c:currentstate s:shard1 r:core_node2 x:currentstate_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7e741d88: rootName = solr_45706, domain = solr.core.currentstate.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.currentstate.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@5a4b633e
[junit4] 2> 470160 INFO (jetty-closer-83-thread-1) [ ] o.a.s.c.Overseer Overseer (id=98676011272503307-127.0.0.1:40241_solr-n_0000000001) closing
[junit4] 2> 470160 INFO (OverseerStateUpdate-98676011272503307-127.0.0.1:40241_solr-n_0000000001) [n:127.0.0.1:40241_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:40241_solr
[junit4] 2> 470160 INFO (coreCloseExecutor-374-thread-3) [n:127.0.0.1:45706_solr c:testcollection s:shard4 r:core_node8 x:testcollection_shard4_replica_n6] o.a.s.c.SolrCore [testcollection_shard4_replica_n6] CLOSING SolrCore org.apache.solr.core.SolrCore@5ac7fc33
[junit4] 2> 470160 INFO (coreCloseExecutor-374-thread-2) [n:127.0.0.1:45706_solr c:currentstate s:shard1 r:core_node2 x:currentstate_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.currentstate.shard1.leader, tag=975361776
[junit4] 2> 470160 INFO (coreCloseExecutor-374-thread-3) [n:127.0.0.1:45706_solr c:testcollection s:shard4 r:core_node8 x:testcollection_shard4_replica_n6] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testcollection.shard4.replica_n6, tag=1523055667
[junit4] 2> 470160 INFO (coreCloseExecutor-374-thread-3) [n:127.0.0.1:45706_solr c:testcollection s:shard4 r:core_node8 x:testcollection_shard4_replica_n6] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2dfe9a25: rootName = solr_45706, domain = solr.core.testcollection.shard4.replica_n6, service url = null, agent id = null] for registry solr.core.testcollection.shard4.replica_n6 / com.codahale.metrics.MetricRegistry@5971e896
[junit4] 2> 470365 INFO (zkCallback-121-thread-3-processing-n:127.0.0.1:45706_solr) [n:127.0.0.1:45706_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 470365 INFO (zkCallback-121-thread-1-processing-n:127.0.0.1:45706_solr) [n:127.0.0.1:45706_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:45706_solr
[junit4] 2> 470366 INFO (coreCloseExecutor-374-thread-1) [n:127.0.0.1:45706_solr c:falsepredicate s:shard3 r:core_node7 x:falsepredicate_shard3_replica_n4] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.falsepredicate.shard3.leader, tag=2111102841
[junit4] 2> 470366 INFO (coreCloseExecutor-374-thread-3) [n:127.0.0.1:45706_solr c:testcollection s:shard4 r:core_node8 x:testcollection_shard4_replica_n6] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testcollection.shard4.leader, tag=1523055667
[junit4] 2> 471898 WARN (zkCallback-103-thread-6-processing-n:127.0.0.1:40241_solr) [n:127.0.0.1:40241_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 471899 INFO (jetty-closer-83-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@3c57760c{/solr,null,UNAVAILABLE}
[junit4] 2> 471943 WARN (zkCallback-121-thread-1-processing-n:127.0.0.1:45706_solr) [n:127.0.0.1:45706_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 471943 INFO (jetty-closer-83-thread-3) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@624ee226{/solr,null,UNAVAILABLE}
[junit4] 2> 471955 ERROR (SUITE-TestCollectionStateWatchers-seed#[69B82F5134D44E35]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 471955 INFO (SUITE-TestCollectionStateWatchers-seed#[69B82F5134D44E35]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:46229 46229
[junit4] 2> 472147 INFO (Thread-58) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:46229 46229
[junit4] 2> 472148 WARN (Thread-58) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 8 /solr/aliases.json
[junit4] 2> 7 /solr/configs/config
[junit4] 2> 6 /solr/security.json
[junit4] 2> 4 /solr/collections/stateformat1/state.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 73 /solr/collections/falsepredicate/state.json
[junit4] 2> 42 /solr/collections/testcollection/state.json
[junit4] 2> 17 /solr/collections/waitforstate/state.json
[junit4] 2> 16 /solr/collections/currentstate/state.json
[junit4] 2> 10 /solr/collections/stateformat1/state.json
[junit4] 2> 8 /solr/clusterprops.json
[junit4] 2> 8 /solr/collections/delayed/state.json
[junit4] 2> 8 /solr/clusterstate.json
[junit4] 2> 6 /solr/collections/tobedeleted/state.json
[junit4] 2> 3 /solr/overseer_elect/election/98676011272503307-127.0.0.1:40241_solr-n_0000000001
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 8 /solr/collections
[junit4] 2> 6 /solr/live_nodes
[junit4] 2> 2 /solr/overseer/queue
[junit4] 2> 2 /solr/overseer/queue-work
[junit4] 2> 2 /solr/overseer/collection-queue-work
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-solrj/test/J1/temp/solr.common.cloud.TestCollectionStateWatchers_69B82F5134D44E35-001
[junit4] 2> Sep 17, 2017 7:29:57 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Lucene70, sim=RandomSimilarity(queryNorm=false): {}, locale=ko-KR, timezone=Africa/Brazzaville
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=210983656,total=435683328
[junit4] 2> NOTE: All tests run in this JVM: [ClientUtilsTest, LessThanEqualToEvaluatorTest, CosineEvaluatorTest, SolrDocumentTest, QueryResponseTest, CoalesceEvaluatorTest, UuidEvaluatorTest, MergeIndexesEmbeddedTest, GreaterThanEqualToEvaluatorTest, PowerEvaluatorTest, TestCloudSolrClientConnections, TangentEvaluatorTest, AddEvaluatorTest, NormalizeEvaluatorTest, TestUpdateRequestCodec, CloudSolrClientTest, FacetFieldTest, TestSolrProperties, HyperbolicSineEvaluatorTest, JdbcDriverTest, RegressionEvaluatorTest, OperationsTest, TestCollectionStateWatchers]
[junit4] Completed [64/145 (1!)] on J1 in 156.95s, 9 tests, 1 failure <<< FAILURES!
[...truncated 45584 lines...]
[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 46 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/46/
7 tests failed.
FAILED: org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.test
Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:54308/jgp/hq/collection1
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:54308/jgp/hq/collection1
at __randomizedtesting.SeedInfo.seed([98D5336665A4B97B:10810CBCCB58D483]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:638)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1121)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:862)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:793)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
at org.apache.solr.client.solrj.SolrClient.deleteByQuery(SolrClient.java:895)
at org.apache.solr.client.solrj.SolrClient.deleteByQuery(SolrClient.java:858)
at org.apache.solr.client.solrj.SolrClient.deleteByQuery(SolrClient.java:873)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.del(AbstractFullDistribZkTestBase.java:888)
at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.test(ChaosMonkeyNothingIsSafeTest.java:135)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:526)
... 54 more
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest
Error Message:
6 threads leaked from SUITE scope at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest: 1) Thread[id=137827, name=zkCallback-32696-thread-4, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=137760, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=138311, name=zkCallback-32696-thread-5, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=137828, name=zkCallback-32696-thread-4-SendThread(127.0.0.1:47905), state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060) 5) Thread[id=138368, name=zkCallback-32696-thread-6, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 6) Thread[id=137829, name=zkCallback-32696-thread-4-EventThread, state=WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 6 threads leaked from SUITE scope at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest:
1) Thread[id=137827, name=zkCallback-32696-thread-4, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=137760, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=138311, name=zkCallback-32696-thread-5, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
4) Thread[id=137828, name=zkCallback-32696-thread-4-SendThread(127.0.0.1:47905), state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)
5) Thread[id=138368, name=zkCallback-32696-thread-6, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
6) Thread[id=137829, name=zkCallback-32696-thread-4-EventThread, state=WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)
at __randomizedtesting.SeedInfo.seed([98D5336665A4B97B]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=137827, name=zkCallback-32696-thread-4, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=138311, name=zkCallback-32696-thread-5, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=137828, name=zkCallback-32696-thread-4-SendThread(127.0.0.1:47905), state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1051) 4) Thread[id=138368, name=zkCallback-32696-thread-6, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=137827, name=zkCallback-32696-thread-4, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=138311, name=zkCallback-32696-thread-5, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=137828, name=zkCallback-32696-thread-4-SendThread(127.0.0.1:47905), state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1051)
4) Thread[id=138368, name=zkCallback-32696-thread-6, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([98D5336665A4B97B]:0)
FAILED: org.apache.solr.cloud.HttpPartitionTest.test
Error Message:
Didn't see all replicas for shard shard1 in c8n_1x2 come up within 90000 ms! ClusterState: { "collMinRf_1x3":{ "pullReplicas":"0", "replicationFactor":"1", "shards":{"shard1":{ "range":"80000000-7fffffff", "state":"active", "replicas":{ "core_node4":{ "core":"collMinRf_1x3_shard1_replica_n3", "base_url":"http://127.0.0.1:53551/_/y", "node_name":"127.0.0.1:53551__%2Fy", "state":"active", "type":"NRT"}, "core_node5":{ "core":"collMinRf_1x3_shard1_replica_n2", "base_url":"http://127.0.0.1:41828/_/y", "node_name":"127.0.0.1:41828__%2Fy", "state":"active", "type":"NRT"}, "core_node6":{ "core":"collMinRf_1x3_shard1_replica_n1", "base_url":"http://127.0.0.1:37574/_/y", "node_name":"127.0.0.1:37574__%2Fy", "state":"active", "type":"NRT", "leader":"true"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"false", "nrtReplicas":"3", "tlogReplicas":"0"}, "collection1":{ "pullReplicas":"0", "replicationFactor":"1", "shards":{ "shard1":{ "range":"80000000-ffffffff", "state":"active", "replicas":{"core_node44":{ "core":"collection1_shard1_replica_n43", "base_url":"http://127.0.0.1:37574/_/y", "node_name":"127.0.0.1:37574__%2Fy", "state":"active", "type":"NRT", "leader":"true"}}}, "shard2":{ "range":"0-7fffffff", "state":"active", "replicas":{ "core_node42":{ "core":"collection1_shard2_replica_n41", "base_url":"http://127.0.0.1:41828/_/y", "node_name":"127.0.0.1:41828__%2Fy", "state":"active", "type":"NRT", "leader":"true"}, "core_node46":{ "core":"collection1_shard2_replica_n45", "base_url":"http://127.0.0.1:53551/_/y", "node_name":"127.0.0.1:53551__%2Fy", "state":"active", "type":"NRT"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"false", "nrtReplicas":"1", "tlogReplicas":"0"}, "control_collection":{ "pullReplicas":"0", "replicationFactor":"1", "shards":{"shard1":{ "range":"80000000-7fffffff", "state":"active", "replicas":{"core_node2":{ "core":"control_collection_shard1_replica_n1", "base_url":"http://127.0.0.1:56322/_/y", "node_name":"127.0.0.1:56322__%2Fy", "state":"active", "type":"NRT", "leader":"true"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"false", "nrtReplicas":"1", "tlogReplicas":"0"}, "c8n_1x2":{ "pullReplicas":"0", "replicationFactor":"1", "shards":{"shard1":{ "range":"80000000-7fffffff", "state":"active", "replicas":{ "core_node3":{ "state":"down", "base_url":"http://127.0.0.1:37574/_/y", "core":"c8n_1x2_shard1_replica_n1", "node_name":"127.0.0.1:37574__%2Fy", "type":"NRT"}, "core_node4":{ "core":"c8n_1x2_shard1_replica_n2", "base_url":"http://127.0.0.1:41828/_/y", "node_name":"127.0.0.1:41828__%2Fy", "state":"active", "type":"NRT", "leader":"true"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"false", "nrtReplicas":"2", "tlogReplicas":"0"}}
Stack Trace:
java.lang.AssertionError: Didn't see all replicas for shard shard1 in c8n_1x2 come up within 90000 ms! ClusterState: {
"collMinRf_1x3":{
"pullReplicas":"0",
"replicationFactor":"1",
"shards":{"shard1":{
"range":"80000000-7fffffff",
"state":"active",
"replicas":{
"core_node4":{
"core":"collMinRf_1x3_shard1_replica_n3",
"base_url":"http://127.0.0.1:53551/_/y",
"node_name":"127.0.0.1:53551__%2Fy",
"state":"active",
"type":"NRT"},
"core_node5":{
"core":"collMinRf_1x3_shard1_replica_n2",
"base_url":"http://127.0.0.1:41828/_/y",
"node_name":"127.0.0.1:41828__%2Fy",
"state":"active",
"type":"NRT"},
"core_node6":{
"core":"collMinRf_1x3_shard1_replica_n1",
"base_url":"http://127.0.0.1:37574/_/y",
"node_name":"127.0.0.1:37574__%2Fy",
"state":"active",
"type":"NRT",
"leader":"true"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"3",
"tlogReplicas":"0"},
"collection1":{
"pullReplicas":"0",
"replicationFactor":"1",
"shards":{
"shard1":{
"range":"80000000-ffffffff",
"state":"active",
"replicas":{"core_node44":{
"core":"collection1_shard1_replica_n43",
"base_url":"http://127.0.0.1:37574/_/y",
"node_name":"127.0.0.1:37574__%2Fy",
"state":"active",
"type":"NRT",
"leader":"true"}}},
"shard2":{
"range":"0-7fffffff",
"state":"active",
"replicas":{
"core_node42":{
"core":"collection1_shard2_replica_n41",
"base_url":"http://127.0.0.1:41828/_/y",
"node_name":"127.0.0.1:41828__%2Fy",
"state":"active",
"type":"NRT",
"leader":"true"},
"core_node46":{
"core":"collection1_shard2_replica_n45",
"base_url":"http://127.0.0.1:53551/_/y",
"node_name":"127.0.0.1:53551__%2Fy",
"state":"active",
"type":"NRT"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"1",
"tlogReplicas":"0"},
"control_collection":{
"pullReplicas":"0",
"replicationFactor":"1",
"shards":{"shard1":{
"range":"80000000-7fffffff",
"state":"active",
"replicas":{"core_node2":{
"core":"control_collection_shard1_replica_n1",
"base_url":"http://127.0.0.1:56322/_/y",
"node_name":"127.0.0.1:56322__%2Fy",
"state":"active",
"type":"NRT",
"leader":"true"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"1",
"tlogReplicas":"0"},
"c8n_1x2":{
"pullReplicas":"0",
"replicationFactor":"1",
"shards":{"shard1":{
"range":"80000000-7fffffff",
"state":"active",
"replicas":{
"core_node3":{
"state":"down",
"base_url":"http://127.0.0.1:37574/_/y",
"core":"c8n_1x2_shard1_replica_n1",
"node_name":"127.0.0.1:37574__%2Fy",
"type":"NRT"},
"core_node4":{
"core":"c8n_1x2_shard1_replica_n2",
"base_url":"http://127.0.0.1:41828/_/y",
"node_name":"127.0.0.1:41828__%2Fy",
"state":"active",
"type":"NRT",
"leader":"true"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"2",
"tlogReplicas":"0"}}
at __randomizedtesting.SeedInfo.seed([98D5336665A4B97B:10810CBCCB58D483]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.ensureAllReplicasAreActive(AbstractFullDistribZkTestBase.java:1996)
at org.apache.solr.cloud.HttpPartitionTest.testRf2(HttpPartitionTest.java:370)
at org.apache.solr.cloud.HttpPartitionTest.test(HttpPartitionTest.java:132)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.TestRandomFlRTGCloud.testRandomizedUpdatesAndRTGs
Error Message:
Error from server at http://127.0.0.1:37917/solr/org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1: Cannot talk to ZooKeeper - Updates are disabled.
Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from server at http://127.0.0.1:37917/solr/org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1: Cannot talk to ZooKeeper - Updates are disabled.
at __randomizedtesting.SeedInfo.seed([98D5336665A4B97B:2609FB9AA2033157]:0)
at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:539)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:993)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:862)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:793)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:173)
at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:138)
at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:152)
at org.apache.solr.cloud.TestRandomFlRTGCloud.addRandomDocument(TestRandomFlRTGCloud.java:333)
at org.apache.solr.cloud.TestRandomFlRTGCloud.assertUpdate(TestRandomFlRTGCloud.java:288)
at org.apache.solr.cloud.TestRandomFlRTGCloud.assertOneIter(TestRandomFlRTGCloud.java:276)
at org.apache.solr.cloud.TestRandomFlRTGCloud.testRandomizedUpdatesAndRTGs(TestRandomFlRTGCloud.java:223)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:37917/solr/org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1: Cannot talk to ZooKeeper - Updates are disabled.
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:627)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:516)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
FAILED: org.apache.solr.cloud.MultiThreadedOCPTest.test
Error Message:
Error from server at https://127.0.0.1:34636: KeeperErrorCode = Session expired for /overseer/collection-map-completed/mn-1
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:34636: KeeperErrorCode = Session expired for /overseer/collection-map-completed/mn-1
at __randomizedtesting.SeedInfo.seed([98D5336665A4B97B:10810CBCCB58D483]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:627)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.getStatusResponse(AbstractFullDistribZkTestBase.java:2241)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.getRequestState(AbstractFullDistribZkTestBase.java:2234)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.getRequestStateAfterCompletion(AbstractFullDistribZkTestBase.java:2216)
at org.apache.solr.cloud.MultiThreadedOCPTest.testParallelCollectionAPICalls(MultiThreadedOCPTest.java:144)
at org.apache.solr.cloud.MultiThreadedOCPTest.test(MultiThreadedOCPTest.java:62)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.security.TestPKIAuthenticationPlugin.test
Error Message:
Stack Trace:
java.lang.NullPointerException
at __randomizedtesting.SeedInfo.seed([98D5336665A4B97B:10810CBCCB58D483]:0)
at org.apache.solr.security.TestPKIAuthenticationPlugin.test(TestPKIAuthenticationPlugin.java:100)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12884 lines...]
[junit4] Suite: org.apache.solr.cloud.TestRandomFlRTGCloud
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.TestRandomFlRTGCloud_98D5336665A4B97B-001/init-core-data-001
[junit4] 2> 2265387 WARN (SUITE-TestRandomFlRTGCloud-seed#[98D5336665A4B97B]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=9 numCloses=9
[junit4] 2> 2265387 INFO (SUITE-TestRandomFlRTGCloud-seed#[98D5336665A4B97B]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 2265389 INFO (SUITE-TestRandomFlRTGCloud-seed#[98D5336665A4B97B]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=client auth uses too much RAM, ssl=NaN, value=NaN, clientAuth=0.0)
[junit4] 2> 2265389 INFO (SUITE-TestRandomFlRTGCloud-seed#[98D5336665A4B97B]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 2265389 INFO (SUITE-TestRandomFlRTGCloud-seed#[98D5336665A4B97B]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.TestRandomFlRTGCloud_98D5336665A4B97B-001/tempDir-001
[junit4] 2> 2265389 INFO (SUITE-TestRandomFlRTGCloud-seed#[98D5336665A4B97B]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 2265454 INFO (Thread-10804) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2265454 INFO (Thread-10804) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 2265593 ERROR (Thread-10804) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2265613 INFO (SUITE-TestRandomFlRTGCloud-seed#[98D5336665A4B97B]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:38246
[junit4] 2> 2265721 INFO (jetty-launcher-1310-thread-1) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 2265919 INFO (jetty-launcher-1310-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1e45c4be{/solr,null,AVAILABLE}
[junit4] 2> 2267467 INFO (jetty-launcher-1310-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2b0b17f3{HTTP/1.1,[http/1.1]}{127.0.0.1:37917}
[junit4] 2> 2267467 INFO (jetty-launcher-1310-thread-1) [ ] o.e.j.s.Server Started @2288094ms
[junit4] 2> 2267467 INFO (jetty-launcher-1310-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37917}
[junit4] 2> 2267468 ERROR (jetty-launcher-1310-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2267468 INFO (jetty-launcher-1310-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.1.0
[junit4] 2> 2267468 INFO (jetty-launcher-1310-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2267468 INFO (jetty-launcher-1310-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 2267468 INFO (jetty-launcher-1310-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-09-15T12:10:25.879Z
[junit4] 2> 2267541 INFO (jetty-launcher-1310-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2267637 INFO (jetty-launcher-1310-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38246/solr
[junit4] 2> 2268159 INFO (jetty-launcher-1310-thread-1) [n:127.0.0.1:37917_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2268167 INFO (jetty-launcher-1310-thread-1) [n:127.0.0.1:37917_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:37917_solr
[junit4] 2> 2268168 INFO (jetty-launcher-1310-thread-1) [n:127.0.0.1:37917_solr ] o.a.s.c.Overseer Overseer (id=98662968454807555-127.0.0.1:37917_solr-n_0000000000) starting
[junit4] 2> 2268357 INFO (jetty-launcher-1310-thread-1) [n:127.0.0.1:37917_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37917_solr
[junit4] 2> 2268359 INFO (OverseerStateUpdate-98662968454807555-127.0.0.1:37917_solr-n_0000000000) [n:127.0.0.1:37917_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2270387 INFO (jetty-launcher-1310-thread-1) [n:127.0.0.1:37917_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37917.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d3d3891
[junit4] 2> 2270507 INFO (jetty-launcher-1310-thread-1) [n:127.0.0.1:37917_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37917.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d3d3891
[junit4] 2> 2270508 INFO (jetty-launcher-1310-thread-1) [n:127.0.0.1:37917_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37917.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d3d3891
[junit4] 2> 2270509 INFO (jetty-launcher-1310-thread-1) [n:127.0.0.1:37917_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.TestRandomFlRTGCloud_98D5336665A4B97B-001/tempDir-001/node1/.
[junit4] 2> 2271122 INFO (SUITE-TestRandomFlRTGCloud-seed#[98D5336665A4B97B]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2271183 INFO (SUITE-TestRandomFlRTGCloud-seed#[98D5336665A4B97B]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:38246/solr ready
[junit4] 2> 2275498 INFO (qtp214146892-14532) [n:127.0.0.1:37917_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&property.config=solrconfig-tlog.xml&property.schema=schema-psuedo-fields.xml&collection.configName=org.apache.solr.cloud.TestRandomFlRTGCloud_config-set&name=org.apache.solr.cloud.TestRandomFlRTGCloud_collection&nrtReplicas=1&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2275501 INFO (OverseerThreadFactory-1079-thread-1-processing-n:127.0.0.1:37917_solr) [n:127.0.0.1:37917_solr ] o.a.s.c.CreateCollectionCmd Create collection org.apache.solr.cloud.TestRandomFlRTGCloud_collection
[junit4] 2> 2275691 INFO (OverseerStateUpdate-98662968454807555-127.0.0.1:37917_solr-n_0000000000) [n:127.0.0.1:37917_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"org.apache.solr.cloud.TestRandomFlRTGCloud_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:37917/solr",
[junit4] 2> "type":"NRT"}
[junit4] 2> 2275921 INFO (qtp214146892-14530) [n:127.0.0.1:37917_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=org.apache.solr.cloud.TestRandomFlRTGCloud_config-set&newCollection=true&collection=org.apache.solr.cloud.TestRandomFlRTGCloud_collection&version=2&replicaType=NRT&property.config=solrconfig-tlog.xml&property.schema=schema-psuedo-fields.xml&coreNodeName=core_node2&name=org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1&action=CREATE&numShards=1&shard=shard1&wt=javabin
[junit4] 2> 2275922 INFO (qtp214146892-14530) [n:127.0.0.1:37917_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2276062 INFO (zkCallback-1317-thread-1-processing-n:127.0.0.1:37917_solr) [n:127.0.0.1:37917_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/org.apache.solr.cloud.TestRandomFlRTGCloud_collection/state.json] for collection [org.apache.solr.cloud.TestRandomFlRTGCloud_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 2276078 INFO (zkCallback-1317-thread-2-processing-n:127.0.0.1:37917_solr) [n:127.0.0.1:37917_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/org.apache.solr.cloud.TestRandomFlRTGCloud_collection/state.json] for collection [org.apache.solr.cloud.TestRandomFlRTGCloud_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 2277067 INFO (qtp214146892-14530) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 2277427 INFO (qtp214146892-14530) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.s.IndexSchema [org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] Schema name=test
[junit4] 2> 2277523 INFO (qtp214146892-14530) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.4 with uniqueid field id
[junit4] 2> 2277625 INFO (qtp214146892-14530) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1' using configuration from collection org.apache.solr.cloud.TestRandomFlRTGCloud_collection, trusted=true
[junit4] 2> 2277637 INFO (qtp214146892-14530) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37917.solr.core.org.apache.solr.cloud.TestRandomFlRTGCloud_collection.shard1.replica_n1' (registry 'solr.core.org.apache.solr.cloud.TestRandomFlRTGCloud_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d3d3891
[junit4] 2> 2277638 INFO (qtp214146892-14530) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2277638 INFO (qtp214146892-14530) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.c.SolrCore [[org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.TestRandomFlRTGCloud_98D5336665A4B97B-001/tempDir-001/node1/org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.TestRandomFlRTGCloud_98D5336665A4B97B-001/tempDir-001/node1/./org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1/data/]
[junit4] 2> 2278022 INFO (qtp214146892-14530) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=29, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 2278035 WARN (qtp214146892-14530) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 2279601 INFO (qtp214146892-14530) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2279601 INFO (qtp214146892-14530) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2279603 INFO (qtp214146892-14530) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2279603 INFO (qtp214146892-14530) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2279701 INFO (qtp214146892-14530) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=35, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 2279705 INFO (qtp214146892-14530) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5f9b31c4[org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] main]
[junit4] 2> 2279710 INFO (qtp214146892-14530) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/org.apache.solr.cloud.TestRandomFlRTGCloud_config-set
[junit4] 2> 2279710 INFO (qtp214146892-14530) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/org.apache.solr.cloud.TestRandomFlRTGCloud_config-set
[junit4] 2> 2279711 INFO (qtp214146892-14530) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2279755 INFO (searcherExecutor-1082-thread-1-processing-n:127.0.0.1:37917_solr x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1 s:shard1 c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection r:core_node2) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.c.SolrCore [org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] Registered new searcher Searcher@5f9b31c4[org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2279762 INFO (qtp214146892-14530) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1578607510209691648
[junit4] 2> 2280077 INFO (zkCallback-1317-thread-2-processing-n:127.0.0.1:37917_solr) [n:127.0.0.1:37917_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/org.apache.solr.cloud.TestRandomFlRTGCloud_collection/state.json] for collection [org.apache.solr.cloud.TestRandomFlRTGCloud_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 2280077 INFO (zkCallback-1317-thread-3-processing-n:127.0.0.1:37917_solr) [n:127.0.0.1:37917_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/org.apache.solr.cloud.TestRandomFlRTGCloud_collection/state.json] for collection [org.apache.solr.cloud.TestRandomFlRTGCloud_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 2280089 INFO (qtp214146892-14530) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2280089 INFO (qtp214146892-14530) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2280089 INFO (qtp214146892-14530) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:37917/solr/org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1/
[junit4] 2> 2280089 INFO (qtp214146892-14530) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 2280089 INFO (qtp214146892-14530) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:37917/solr/org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 2280089 INFO (qtp214146892-14530) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 2280153 INFO (qtp214146892-14530) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:37917/solr/org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1/ shard1
[junit4] 2> 2280257 INFO (zkCallback-1317-thread-3-processing-n:127.0.0.1:37917_solr) [n:127.0.0.1:37917_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/org.apache.solr.cloud.TestRandomFlRTGCloud_collection/state.json] for collection [org.apache.solr.cloud.TestRandomFlRTGCloud_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 2280258 INFO (zkCallback-1317-thread-2-processing-n:127.0.0.1:37917_solr) [n:127.0.0.1:37917_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/org.apache.solr.cloud.TestRandomFlRTGCloud_collection/state.json] for collection [org.apache.solr.cloud.TestRandomFlRTGCloud_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 2280293 INFO (qtp214146892-14530) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2280313 INFO (qtp214146892-14530) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=org.apache.solr.cloud.TestRandomFlRTGCloud_config-set&newCollection=true&collection=org.apache.solr.cloud.TestRandomFlRTGCloud_collection&version=2&replicaType=NRT&property.config=solrconfig-tlog.xml&property.schema=schema-psuedo-fields.xml&coreNodeName=core_node2&name=org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1&action=CREATE&numShards=1&shard=shard1&wt=javabin} status=0 QTime=4391
[junit4] 2> 2280370 INFO (qtp214146892-14532) [n:127.0.0.1:37917_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 2280401 INFO (zkCallback-1317-thread-3-processing-n:127.0.0.1:37917_solr) [n:127.0.0.1:37917_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/org.apache.solr.cloud.TestRandomFlRTGCloud_collection/state.json] for collection [org.apache.solr.cloud.TestRandomFlRTGCloud_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 2280401 INFO (zkCallback-1317-thread-2-processing-n:127.0.0.1:37917_solr) [n:127.0.0.1:37917_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/org.apache.solr.cloud.TestRandomFlRTGCloud_collection/state.json] for collection [org.apache.solr.cloud.TestRandomFlRTGCloud_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 2281373 INFO (qtp214146892-14532) [n:127.0.0.1:37917_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&property.config=solrconfig-tlog.xml&property.schema=schema-psuedo-fields.xml&collection.configName=org.apache.solr.cloud.TestRandomFlRTGCloud_config-set&name=org.apache.solr.cloud.TestRandomFlRTGCloud_collection&nrtReplicas=1&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=5874
[junit4] 2> 2281373 INFO (SUITE-TestRandomFlRTGCloud-seed#[98D5336665A4B97B]-worker) [ ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: org.apache.solr.cloud.TestRandomFlRTGCloud_collection failOnTimeout:true timeout (sec):330
[junit4] 1> -
[junit4] 1> replica:core_node2 rstate:active live:true
[junit4] 1> no one is recoverying
[junit4] 2> 2281377 INFO (SUITE-TestRandomFlRTGCloud-seed#[98D5336665A4B97B]-worker) [ ] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: org.apache.solr.cloud.TestRandomFlRTGCloud_collection
[junit4] 2> 2281509 INFO (OverseerCollectionConfigSetProcessor-98662968454807555-127.0.0.1:37917_solr-n_0000000000) [n:127.0.0.1:37917_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2281605 INFO (TEST-TestRandomFlRTGCloud.testCoverage-seed#[98D5336665A4B97B]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testCoverage
[junit4] 2> 2281606 INFO (TEST-TestRandomFlRTGCloud.testCoverage-seed#[98D5336665A4B97B]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testCoverage
[junit4] 2> 2281801 INFO (TEST-TestRandomFlRTGCloud.testRandomizedUpdatesAndRTGs-seed#[98D5336665A4B97B]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testRandomizedUpdatesAndRTGs
[junit4] 2> 2281801 INFO (TEST-TestRandomFlRTGCloud.testRandomizedUpdatesAndRTGs-seed#[98D5336665A4B97B]) [ ] o.a.s.c.TestRandomFlRTGCloud Starting 20222 iters by seeding 21 of 409 max docs
[junit4] 2> 2281801 INFO (TEST-TestRandomFlRTGCloud.testRandomizedUpdatesAndRTGs-seed#[98D5336665A4B97B]) [ ] o.a.s.c.TestRandomFlRTGCloud ADD: 0 = SolrInputDocument(fields: [id=0, aaa_i=1262762029, bbb_i=1946580244, ccc_s=ervsvp, ddd_s=rvp, eee_s=dztibo, fff_s=jpw, ggg_s=, hhh_s=ghcxoh, geo_1_srpt=POINT( 42 36 ), geo_2_srpt=POINT( 42 36 ), next_2_ids_ss=[1, 2], axx_i=-1156244336, ayy_i=-1350557741, azz_s=vwlexian])
[junit4] 2> 2281881 INFO (qtp214146892-14534) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.s.AbstractSpatialPrefixTreeFieldType location_rpt{class=org.apache.solr.schema.SpatialRecursivePrefixTreeFieldType,analyzer=org.apache.solr.schema.FieldType$DefaultAnalyzer,args={geo=true, maxDistErr=0.001, distErrPct=0.025, distanceUnits=kilometers, class=solr.SpatialRecursivePrefixTreeFieldType}} strat: RecursivePrefixTreeStrategy(SPG:(GeohashPrefixTree(maxLevels:11,ctx:SpatialContext{geo=true, calculator=Haversine, worldBounds=Rect(minX=-180.0,maxX=180.0,minY=-90.0,maxY=90.0)})),pruneLeafyBranches) maxLevels: 11
[junit4] 2> 2281881 INFO (qtp214146892-14534) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.s.AbstractSpatialPrefixTreeFieldType location_rpt{class=org.apache.solr.schema.SpatialRecursivePrefixTreeFieldType,analyzer=org.apache.solr.schema.FieldType$DefaultAnalyzer,args={geo=true, maxDistErr=0.001, distErrPct=0.025, distanceUnits=kilometers, class=solr.SpatialRecursivePrefixTreeFieldType}} strat: RecursivePrefixTreeStrategy(SPG:(GeohashPrefixTree(maxLevels:11,ctx:SpatialContext{geo=true, calculator=Haversine, worldBounds=Rect(minX=-180.0,maxX=180.0,minY=-90.0,maxY=90.0)})),pruneLeafyBranches) maxLevels: 11
[junit4] 2> 2281990 INFO (qtp214146892-14534) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[0 (1578607512427429888)]} 0 148
[junit4] 2> 2281993 INFO (TEST-TestRandomFlRTGCloud.testRandomizedUpdatesAndRTGs-seed#[98D5336665A4B97B]) [ ] o.a.s.c.TestRandomFlRTGCloud ADD: 1 = SolrInputDocument(fields: [id=1, aaa_i=-536039728, bbb_i=-888385529, ccc_s=bxewr, ddd_s=cyoi, eee_s=z, fff_s=rybgqrn, ggg_s=v, hhh_s=qf, geo_1_srpt=POINT( 42 -3 ), geo_2_srpt=POINT( 42 10 ), next_2_ids_ss=[2, 3], axx_i=290032420, ayy_i=449856086, azz_s=])
[junit4] 2> 2282030 INFO (qtp214146892-14535) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[1 (1578607512550113280)]} 0 36
[junit4] 2> 2282030 INFO (TEST-TestRandomFlRTGCloud.testRandomizedUpdatesAndRTGs-seed#[98D5336665A4B97B]) [ ] o.a.s.c.TestRandomFlRTGCloud ADD: 2 = SolrInputDocument(fields: [id=2, aaa_i=-585361730, bbb_i=1780447992, ccc_s=mw, ddd_s=s, eee_s=mqcvy, fff_s=kkkch, ggg_s=r, hhh_s=tu, geo_1_srpt=POINT( 42 10 ), geo_2_srpt=POINT( 42 -16 ), next_2_ids_ss=[3, 4], axx_i=1689444516, ayy_i=-1372306970, azz_s=sg])
[junit4] 2> 2282097 INFO (qtp214146892-14536) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[2 (1578607512591007744)]} 0 64
[junit4] 2> 2282098 INFO (TEST-TestRandomFlRTGCloud.testRandomizedUpdatesAndRTGs-seed#[98D5336665A4B97B]) [ ] o.a.s.c.TestRandomFlRTGCloud ADD: 3 = SolrInputDocument(fields: [id=3, aaa_i=682805613, bbb_i=-397821396, ccc_s=gmekbbmym, ddd_s=tbrzncbdl, eee_s=coqgpiw, fff_s=mhbadc, ggg_s=uh, hhh_s=ni, geo_1_srpt=POINT( 42 10 ), geo_2_srpt=POINT( 42 36 ), next_2_ids_ss=[4, 5], axx_i=-371394236, ayy_i=-2025215012, azz_s=rthjoh])
[junit4] 2> 2282100 INFO (qtp214146892-14533) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[3 (1578607512660213760)]} 0 1
[junit4] 2> 2282105 INFO (TEST-TestRandomFlRTGCloud.testRandomizedUpdatesAndRTGs-seed#[98D5336665A4B97B]) [ ] o.a.s.c.TestRandomFlRTGCloud ADD: 4 = SolrInputDocument(fields: [id=4, aaa_i=-270981193, bbb_i=292489040, ccc_s=jnjirlymvr, ddd_s=vslefwu, eee_s=z, fff_s=hvvovgd, ggg_s=jg, hhh_s=guyeb, geo_1_srpt=POINT( 42 -42 ), geo_2_srpt=POINT( 42 36 ), next_2_ids_ss=[5, 6], axx_i=1189417416, ayy_i=-1954777151, azz_s=ki])
[junit4] 2> 2282132 INFO (qtp214146892-14530) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[4 (1578607512693768192)]} 0 0
[junit4] 2> 2282143 INFO (TEST-TestRandomFlRTGCloud.testRandomizedUpdatesAndRTGs-seed#[98D5336665A4B97B]) [ ] o.a.s.c.TestRandomFlRTGCloud ADD: 5 = SolrInputDocument(fields: [id=5, aaa_i=1888855830, bbb_i=-647420089, ccc_s=f, ddd_s=yiwn, eee_s=oo, fff_s=rklldqeb, ggg_s=ems, hhh_s=wgawpu, geo_1_srpt=POINT( 42 -16 ), geo_2_srpt=POINT( 42 23 ), next_2_ids_ss=[6, 7], axx_i=750626493, ayy_i=1029575000, azz_s=ezhk])
[junit4] 2> 2282223 INFO (qtp214146892-14530) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[5 (1578607512707399680)]} 0 79
[junit4] 2> 2282224 INFO (TEST-TestRandomFlRTGCloud.testRandomizedUpdatesAndRTGs-seed#[98D5336665A4B97B]) [ ] o.a.s.c.TestRandomFlRTGCloud ADD: 6 = SolrInputDocument(fields: [id=6, aaa_i=2071835192, bbb_i=-1513859993, ccc_s=jupv, ddd_s=xkoh, eee_s=aecadmgi, fff_s=bvk, ggg_s=igvbswmdo, hhh_s=lltpdrxg, geo_1_srpt=POINT( 42 10 ), geo_2_srpt=POINT( 42 62 ), next_2_ids_ss=[7, 8], axx_i=-609825404, ayy_i=935747949, azz_s=unhylwz])
[junit4] 2> 2282266 INFO (qtp214146892-14534) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[6 (1578607512834277376)]} 0 41
[junit4] 2> 2282273 INFO (TEST-TestRandomFlRTGCloud.testRandomizedUpdatesAndRTGs-seed#[98D5336665A4B97B]) [ ] o.a.s.c.TestRandomFlRTGCloud ADD: 7 = SolrInputDocument(fields: [id=7, aaa_i=431411880, bbb_i=1518421361, ccc_s=xk, ddd_s=ntlbebcy, eee_s=nmbe, fff_s=oowtvcel, ggg_s=yfsze, hhh_s=, geo_1_srpt=POINT( 42 -29 ), geo_2_srpt=POINT( 42 -3 ), next_2_ids_ss=[8, 9], axx_i=-1453679907, ayy_i=932645737, azz_s=iceeol])
[junit4] 2> 2282354 INFO (qtp214146892-14535) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[7 (1578607512926552064)]} 0 0
[junit4] 2> 2282361 INFO (TEST-TestRandomFlRTGCloud.testRandomizedUpdatesAndRTGs-seed#[98D5336665A4B97B]) [ ] o.a.s.c.TestRandomFlRTGCloud ADD: 8 = SolrInputDocument(fields: [id=8, aaa_i=131835052, bbb_i=1798003715, ccc_s=lx, ddd_s=rqdfthxoa, eee_s=khqcifffa, fff_s=bb, ggg_s=f, hhh_s=icgxvgdnn, geo_1_srpt=POINT( 42 -42 ), geo_2_srpt=POINT( 42 62 ), next_2_ids_ss=[9, 10], axx_i=-876202257, ayy_i=1398774898, azz_s=wafavwqdii])
[junit4] 2> 2282420 INFO (qtp214146892-14535) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[8 (1578607512939134976)]} 0 54
[junit4] 2> 2282441 INFO (TEST-TestRandomFlRTGCloud.testRandomizedUpdatesAndRTGs-seed#[98D5336665A4B97B]) [ ] o.a.s.c.TestRandomFlRTGCloud ADD: 9 = SolrInputDocument(fields: [id=9, aaa_i=630962163, bbb_i=2130422014, ccc_s=v, ddd_s=kmefkw, eee_s=kwlhv, fff_s=osxa, ggg_s=abwd, hhh_s=rl, geo_1_srpt=POINT( 42 49 ), geo_2_srpt=POINT( 42 62 ), next_2_ids_ss=[10, 11], axx_i=-1511134604, ayy_i=58054521, azz_s=fven])
[junit4] 2> 2282477 INFO (qtp214146892-14529) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[9 (1578607513019875328)]} 0 35
[junit4] 2> 2282478 INFO (TEST-TestRandomFlRTGCloud.testRandomizedUpdatesAndRTGs-seed#[98D5336665A4B97B]) [ ] o.a.s.c.TestRandomFlRTGCloud ADD: 10 = SolrInputDocument(fields: [id=10, aaa_i=719949751, bbb_i=1773547937, ccc_s=gfghgy, ddd_s=duqi, eee_s=, fff_s=ppsnzdek, ggg_s=llmrq, hhh_s=aesqstil, geo_1_srpt=POINT( 42 -16 ), geo_2_srpt=POINT( 42 23 ), next_2_ids_ss=[11, 12], axx_i=-1321548376, ayy_i=-1019134824, azz_s=lpjgnpkk])
[junit4] 2> 2282480 INFO (qtp214146892-14533) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[10 (1578607513058672640)]} 0 0
[junit4] 2> 2282480 INFO (TEST-TestRandomFlRTGCloud.testRandomizedUpdatesAndRTGs-seed#[98D5336665A4B97B]) [ ] o.a.s.c.TestRandomFlRTGCloud ADD: 11 = SolrInputDocument(fields: [id=11, aaa_i=484479121, bbb_i=-658530674, ccc_s=pwyksinw, ddd_s=xl, eee_s=mkaek, fff_s=i, ggg_s=hlgolafev, hhh_s=kqwxukiflm, geo_1_srpt=POINT( 42 36 ), geo_2_srpt=POINT( 42 -29 ), next_2_ids_ss=[12, 13], axx_i=-705678315, ayy_i=2117131670, azz_s=jh])
[junit4] 2> 2282502 INFO (qtp214146892-14532) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[11 (1578607513060769792)]} 0 21
[junit4] 2> 2282502 INFO (TEST-TestRandomFlRTGCloud.testRandomizedUpdatesAndRTGs-seed#[98D5336665A4B97B]) [ ] o.a.s.c.TestRandomFlRTGCloud ADD: 12 = SolrInputDocument(fields: [id=12, aaa_i=-1983393623, bbb_i=819463938, ccc_s=gtaomd, ddd_s=xed, eee_s=zigruzz, fff_s=vpfgnsr, ggg_s=mtgpxaialn, hhh_s=kfzq, geo_1_srpt=POINT( 42 23 ), geo_2_srpt=POINT( 42 36 ), next_2_ids_ss=[13, 14], axx_i=-1620683947, ayy_i=-2040052954, azz_s=zzleuoz])
[junit4] 2> 2282503 INFO (qtp214146892-14536) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[12 (1578607513083838464)]} 0 0
[junit4] 2> 2282503 INFO (TEST-TestRandomFlRTGCloud.testRandomizedUpdatesAndRTGs-seed#[98D5336665A4B97B]) [ ] o.a.s.c.TestRandomFlRTGCloud ADD: 13 = SolrInputDocument(fields: [id=13, aaa_i=576005911, bbb_i=1104987294, ccc_s=, ddd_s=piu, eee_s=enyw, fff_s=jiyqghe, ggg_s=, hhh_s=oml, geo_1_srpt=POINT( 42 -29 ), geo_2_srpt=POINT( 42 62 ), next_2_ids_ss=[14, 15], axx_i=-1980861974, ayy_i=-1660093415, azz_s=uimjtki])
[junit4] 2> 2282504 INFO (qtp214146892-14536) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[13 (1578607513084887040)]} 0 0
[junit4] 2> 2282504 INFO (TEST-TestRandomFlRTGCloud.testRandomizedUpdatesAndRTGs-seed#[98D5336665A4B97B]) [ ] o.a.s.c.TestRandomFlRTGCloud ADD: 14 = SolrInputDocument(fields: [id=14, aaa_i=-445963685, bbb_i=876126169, ccc_s=ricsc, ddd_s=nqkcdxk, eee_s=tcibaaah, fff_s=btygezob, ggg_s=cbqdhdeg, hhh_s=cxiapu, geo_1_srpt=POINT( 42 -29 ), geo_2_srpt=POINT( 42 10 ), next_2_ids_ss=[15, 16], axx_i=-1157358402, ayy_i=-1857698442, azz_s=j])
[junit4] 2> 2282534 INFO (qtp214146892-14536) [n:127.0.0.1:37917_solr c:org.apache.solr.cloud.TestRandomFlRTGCloud_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestRandomFlRTGCloud_collection_shard1_replica_n1] o.a.s.u.p.LogUpdateProce
[...truncated too long message...]
eReloadTest, SignatureUpdateProcessorFactoryTest, SuggesterFSTTest, SuggesterTest, TestStressLucene, SpatialFilterTest, SuggesterWFSTTest, TestUpdate, TestAtomicUpdateErrorCases, SortByFunctionTest, TestSurroundQueryParser, DocumentAnalysisRequestHandlerTest, TestQueryTypes, PrimitiveFieldTypeTest, XmlUpdateRequestHandlerTest, MoreLikeThisHandlerTest, RequiredFieldsTest, FastVectorHighlighterTest, LoggingHandlerTest, TestJmxIntegration, MBeansHandlerTest, TestQuerySenderListener, CopyFieldTest, TestSolrDeletionPolicy2, SampleTest, TestSearchPerf, TestConfig, TestFuzzyAnalyzedSuggestions, ExternalFileFieldSortTest, DirectSolrConnectionTest, TestCharFilters, TestCodecSupport, TestPerFieldSimilarity, TestLMJelinekMercerSimilarityFactory, OpenExchangeRatesOrgProviderTest, TestFastLRUCache, ChaosMonkeyNothingIsSafeTest]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=ChaosMonkeyNothingIsSafeTest -Dtests.seed=98D5336665A4B97B -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=lt-LT -Dtests.timezone=Asia/Bishkek -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J2 | ChaosMonkeyNothingIsSafeTest (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 6 threads leaked from SUITE scope at org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest:
[junit4] > 1) Thread[id=137827, name=zkCallback-32696-thread-4, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 2) Thread[id=137760, name=Connection evictor, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 3) Thread[id=138311, name=zkCallback-32696-thread-5, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 4) Thread[id=137828, name=zkCallback-32696-thread-4-SendThread(127.0.0.1:47905), state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)
[junit4] > 5) Thread[id=138368, name=zkCallback-32696-thread-6, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 6) Thread[id=137829, name=zkCallback-32696-thread-4-EventThread, state=WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)
[junit4] > at __randomizedtesting.SeedInfo.seed([98D5336665A4B97B]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=137827, name=zkCallback-32696-thread-4, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 2) Thread[id=138311, name=zkCallback-32696-thread-5, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 3) Thread[id=137828, name=zkCallback-32696-thread-4-SendThread(127.0.0.1:47905), state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1051)
[junit4] > 4) Thread[id=138368, name=zkCallback-32696-thread-6, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([98D5336665A4B97B]:0)
[junit4] Completed [492/731 (5!)] on J2 in 242.16s, 1 test, 3 errors <<< FAILURES!
[...truncated 46975 lines...]
[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 45 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/45/
13 tests failed.
FAILED: org.apache.solr.cloud.TestTlogReplica.testCreateDelete {seed=[F05616602695BA2D:5F105BD0E4A78931]}
Error Message:
KeeperErrorCode = Session expired for /
Stack Trace:
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /
at __randomizedtesting.SeedInfo.seed([F05616602695BA2D:5F105BD0E4A78931]:0)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1212)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:357)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:354)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:354)
at org.apache.solr.common.cloud.SolrZkClient.printLayout(SolrZkClient.java:606)
at org.apache.solr.common.cloud.SolrZkClient.printLayoutToStream(SolrZkClient.java:651)
at org.apache.solr.common.cloud.SolrZkClient.printLayoutToStdOut(SolrZkClient.java:646)
at org.apache.solr.cloud.TestTlogReplica.testCreateDelete(TestTlogReplica.java:218)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.hdfs.HdfsBasicDistributedZk2Test.test
Error Message:
expected:<69> but was:<68>
Stack Trace:
java.lang.AssertionError: expected:<69> but was:<68>
at __randomizedtesting.SeedInfo.seed([F05616602695BA2D:780229BA8869D7D5]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.BasicDistributedZk2Test.brindDownShardIndexSomeDocsAndRecover(BasicDistributedZk2Test.java:337)
at org.apache.solr.cloud.BasicDistributedZk2Test.test(BasicDistributedZk2Test.java:108)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.hdfs.HdfsBasicDistributedZkTest.test
Error Message:
Error from server at http://127.0.0.1:44353: ADDREPLICA failed to create replica
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:44353: ADDREPLICA failed to create replica
at __randomizedtesting.SeedInfo.seed([F05616602695BA2D:780229BA8869D7D5]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:627)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1121)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:862)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:793)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195)
at org.apache.solr.cloud.BasicDistributedZkTest.testANewCollectionInOneInstanceWithManualShardAssignement(BasicDistributedZkTest.java:801)
at org.apache.solr.cloud.BasicDistributedZkTest.test(BasicDistributedZkTest.java:374)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsBasicDistributedZkTest
Error Message:
3 threads leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsBasicDistributedZkTest: 1) Thread[id=26712, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HdfsBasicDistributedZkTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=26692, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HdfsBasicDistributedZkTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=26701, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HdfsBasicDistributedZkTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 3 threads leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsBasicDistributedZkTest:
1) Thread[id=26712, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HdfsBasicDistributedZkTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=26692, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HdfsBasicDistributedZkTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=26701, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HdfsBasicDistributedZkTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([F05616602695BA2D]:0)
FAILED: org.apache.solr.client.solrj.io.stream.StreamExpressionTest.testParallelExecutorStream
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([C4BB7318CC98FC59:79AC0601F5B4C104]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.client.solrj.io.stream.StreamExpressionTest.testParallelExecutorStream(StreamExpressionTest.java:7272)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.client.solrj.io.stream.StreamExpressionTest
Error Message:
10 threads leaked from SUITE scope at org.apache.solr.client.solrj.io.stream.StreamExpressionTest: 1) Thread[id=3458, name=zkCallback-1404-thread-3, state=TIMED_WAITING, group=TGRP-StreamExpressionTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=2987, name=Connection evictor, state=TIMED_WAITING, group=TGRP-StreamExpressionTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=2983, name=zkCallback-1404-thread-1, state=TIMED_WAITING, group=TGRP-StreamExpressionTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=3465, name=zkCallback-1404-thread-4-SendThread(127.0.0.1:45246), state=TIMED_WAITING, group=TGRP-StreamExpressionTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060) 5) Thread[id=2986, name=Connection evictor, state=TIMED_WAITING, group=TGRP-StreamExpressionTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 6) Thread[id=3463, name=zkCallback-1404-thread-4, state=TIMED_WAITING, group=TGRP-StreamExpressionTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 7) Thread[id=2980, name=Connection evictor, state=TIMED_WAITING, group=TGRP-StreamExpressionTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 8) Thread[id=4089, name=zkCallback-1404-thread-5, state=TIMED_WAITING, group=TGRP-StreamExpressionTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 9) Thread[id=3466, name=zkCallback-1404-thread-4-EventThread, state=WAITING, group=TGRP-StreamExpressionTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501) 10) Thread[id=3457, name=zkCallback-1404-thread-2, state=TIMED_WAITING, group=TGRP-StreamExpressionTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 10 threads leaked from SUITE scope at org.apache.solr.client.solrj.io.stream.StreamExpressionTest:
1) Thread[id=3458, name=zkCallback-1404-thread-3, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=2987, name=Connection evictor, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=2983, name=zkCallback-1404-thread-1, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
4) Thread[id=3465, name=zkCallback-1404-thread-4-SendThread(127.0.0.1:45246), state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)
5) Thread[id=2986, name=Connection evictor, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
6) Thread[id=3463, name=zkCallback-1404-thread-4, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
7) Thread[id=2980, name=Connection evictor, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
8) Thread[id=4089, name=zkCallback-1404-thread-5, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
9) Thread[id=3466, name=zkCallback-1404-thread-4-EventThread, state=WAITING, group=TGRP-StreamExpressionTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)
10) Thread[id=3457, name=zkCallback-1404-thread-2, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([C4BB7318CC98FC59]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.client.solrj.io.stream.StreamExpressionTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=3458, name=zkCallback-1404-thread-3, state=TIMED_WAITING, group=TGRP-StreamExpressionTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=2983, name=zkCallback-1404-thread-1, state=TIMED_WAITING, group=TGRP-StreamExpressionTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=3465, name=zkCallback-1404-thread-4-SendThread(127.0.0.1:45246), state=TIMED_WAITING, group=TGRP-StreamExpressionTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060) 4) Thread[id=3463, name=zkCallback-1404-thread-4, state=TIMED_WAITING, group=TGRP-StreamExpressionTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=4089, name=zkCallback-1404-thread-5, state=TIMED_WAITING, group=TGRP-StreamExpressionTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 6) Thread[id=3457, name=zkCallback-1404-thread-2, state=TIMED_WAITING, group=TGRP-StreamExpressionTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=3458, name=zkCallback-1404-thread-3, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=2983, name=zkCallback-1404-thread-1, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=3465, name=zkCallback-1404-thread-4-SendThread(127.0.0.1:45246), state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)
4) Thread[id=3463, name=zkCallback-1404-thread-4, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
5) Thread[id=4089, name=zkCallback-1404-thread-5, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
6) Thread[id=3457, name=zkCallback-1404-thread-2, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([C4BB7318CC98FC59]:0)
FAILED: org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([88311BF75969D14]:0)
FAILED: junit.framework.TestSuite.org.apache.lucene.spatial3d.TestGeo3DPoint
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([88311BF75969D14]:0)
FAILED: org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest.test
Error Message:
The Monkey ran for over 45 seconds and no jetties were stopped - this is worth investigating!
Stack Trace:
java.lang.AssertionError: The Monkey ran for over 45 seconds and no jetties were stopped - this is worth investigating!
at __randomizedtesting.SeedInfo.seed([F05616602695BA2D:780229BA8869D7D5]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.ChaosMonkey.stopTheMonkey(ChaosMonkey.java:587)
at org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest.test(ChaosMonkeySafeLeaderWithPullReplicasTest.java:174)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest
Error Message:
8 threads leaked from SUITE scope at org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest: 1) Thread[id=133221, name=StoppableCommitThread-SendThread(127.0.0.1:45020), state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060) 2) Thread[id=133360, name=zkCallback-32593-thread-2, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=133137, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at java.lang.Thread.sleep(Native Method) at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135) 4) Thread[id=133361, name=zkCallback-32593-thread-3, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=133139, name=StoppableCommitThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at java.lang.Thread.sleep(Native Method) at org.apache.solr.cloud.StoppableCommitThread.run(StoppableCommitThread.java:55) 6) Thread[id=133138, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at java.lang.Thread.sleep(Native Method) at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135) 7) Thread[id=133223, name=zkCallback-32593-thread-1, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 8) Thread[id=133222, name=StoppableCommitThread-EventThread, state=WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 8 threads leaked from SUITE scope at org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest:
1) Thread[id=133221, name=StoppableCommitThread-SendThread(127.0.0.1:45020), state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)
2) Thread[id=133360, name=zkCallback-32593-thread-2, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=133137, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135)
4) Thread[id=133361, name=zkCallback-32593-thread-3, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
5) Thread[id=133139, name=StoppableCommitThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.solr.cloud.StoppableCommitThread.run(StoppableCommitThread.java:55)
6) Thread[id=133138, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135)
7) Thread[id=133223, name=zkCallback-32593-thread-1, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
8) Thread[id=133222, name=StoppableCommitThread-EventThread, state=WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)
at __randomizedtesting.SeedInfo.seed([F05616602695BA2D]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=133221, name=StoppableCommitThread-SendThread(127.0.0.1:45020), state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1051) 2) Thread[id=133360, name=zkCallback-32593-thread-2, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=133137, name=StoppableIndexingThread, state=BLOCKED, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at java.lang.Throwable.printStackTrace(Throwable.java:643) at java.lang.Throwable.printStackTrace(Throwable.java:634) at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:121) 4) Thread[id=133361, name=zkCallback-32593-thread-3, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=133138, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at java.util.Collections$SetFromMap.add(Collections.java:5461) at java.lang.Throwable.printStackTrace(Throwable.java:651) at java.lang.Throwable.printStackTrace(Throwable.java:643) at java.lang.Throwable.printStackTrace(Throwable.java:634) at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:121) 6) Thread[id=133223, name=zkCallback-32593-thread-1, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=133221, name=StoppableCommitThread-SendThread(127.0.0.1:45020), state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1051)
2) Thread[id=133360, name=zkCallback-32593-thread-2, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=133137, name=StoppableIndexingThread, state=BLOCKED, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at java.lang.Throwable.printStackTrace(Throwable.java:643)
at java.lang.Throwable.printStackTrace(Throwable.java:634)
at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:121)
4) Thread[id=133361, name=zkCallback-32593-thread-3, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
5) Thread[id=133138, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at java.util.Collections$SetFromMap.add(Collections.java:5461)
at java.lang.Throwable.printStackTrace(Throwable.java:651)
at java.lang.Throwable.printStackTrace(Throwable.java:643)
at java.lang.Throwable.printStackTrace(Throwable.java:634)
at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:121)
6) Thread[id=133223, name=zkCallback-32593-thread-1, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([F05616602695BA2D]:0)
FAILED: org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_dv
Error Message:
java.lang.RuntimeException: Error from server at http://127.0.0.1:33503/solr/test_col: Async exception during distributed update: Error from server at http://127.0.0.1:45061/solr/test_col_shard5_replica_n15: Server Error request: http://127.0.0.1:45061/solr/test_col_shard5_replica_n15/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A33503%2Fsolr%2Ftest_col_shard4_replica_n12%2F&wt=javabin&version=2 Remote error message: Failed synchronous update on shard StdNode: http://127.0.0.1:33371/solr/test_col_shard5_replica_n17/ update: org.apache.solr.client.solrj.request.UpdateRequest@6b3efac2
Stack Trace:
java.util.concurrent.ExecutionException: java.lang.RuntimeException: Error from server at http://127.0.0.1:33503/solr/test_col: Async exception during distributed update: Error from server at http://127.0.0.1:45061/solr/test_col_shard5_replica_n15: Server Error
request: http://127.0.0.1:45061/solr/test_col_shard5_replica_n15/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A33503%2Fsolr%2Ftest_col_shard4_replica_n12%2F&wt=javabin&version=2
Remote error message: Failed synchronous update on shard StdNode: http://127.0.0.1:33371/solr/test_col_shard5_replica_n17/ update: org.apache.solr.client.solrj.request.UpdateRequest@6b3efac2
at __randomizedtesting.SeedInfo.seed([F05616602695BA2D:C6427426ACC8803C]:0)
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.checkField(TestStressCloudBlindAtomicUpdates.java:283)
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_dv(TestStressCloudBlindAtomicUpdates.java:195)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: Error from server at http://127.0.0.1:33503/solr/test_col: Async exception during distributed update: Error from server at http://127.0.0.1:45061/solr/test_col_shard5_replica_n15: Server Error
request: http://127.0.0.1:45061/solr/test_col_shard5_replica_n15/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A33503%2Fsolr%2Ftest_col_shard4_replica_n12%2F&wt=javabin&version=2
Remote error message: Failed synchronous update on shard StdNode: http://127.0.0.1:33371/solr/test_col_shard5_replica_n17/ update: org.apache.solr.client.solrj.request.UpdateRequest@6b3efac2
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.run(TestStressCloudBlindAtomicUpdates.java:411)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:33503/solr/test_col: Async exception during distributed update: Error from server at http://127.0.0.1:45061/solr/test_col_shard5_replica_n15: Server Error
request: http://127.0.0.1:45061/solr/test_col_shard5_replica_n15/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A33503%2Fsolr%2Ftest_col_shard4_replica_n12%2F&wt=javabin&version=2
Remote error message: Failed synchronous update on shard StdNode: http://127.0.0.1:33371/solr/test_col_shard5_replica_n17/ update: org.apache.solr.client.solrj.request.UpdateRequest@6b3efac2
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:627)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195)
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.doRandomAtomicUpdate(TestStressCloudBlindAtomicUpdates.java:370)
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.run(TestStressCloudBlindAtomicUpdates.java:399)
... 6 more
Build Log:
[...truncated 10627 lines...]
[junit4] Suite: org.apache.lucene.spatial3d.TestGeo3DPoint
[junit4] 2> sep 14, 2017 2:13:43 AM com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
[junit4] 2> WARNING: Suite execution timed out: org.apache.lucene.spatial3d.TestGeo3DPoint
[junit4] 2> 1) Thread[id=11, name=SUITE-TestGeo3DPoint-seed#[88311BF75969D14], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at java.lang.Thread.getStackTrace(Thread.java:1559)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:696)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:693)
[junit4] 2> at java.security.AccessController.doPrivileged(Native Method)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getStackTrace(ThreadLeakControl.java:693)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getThreadsWithTraces(ThreadLeakControl.java:709)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:689)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:65)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:415)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:705)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:139)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:626)
[junit4] 2> 2) Thread[id=9, name=JUnit4-serializer-daemon, state=TIMED_WAITING, group=main]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:50)
[junit4] 2> 3) Thread[id=12, name=TEST-TestGeo3DPoint.testRandomBig-seed#[88311BF75969D14], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at org.apache.lucene.spatial3d.geom.Plane.findIntersections(Plane.java:650)
[junit4] 2> at org.apache.lucene.spatial3d.geom.StandardXYZSolid.<init>(StandardXYZSolid.java:183)
[junit4] 2> at org.apache.lucene.spatial3d.geom.XYZSolidFactory.makeXYZSolid(XYZSolidFactory.java:64)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.verify(TestGeo3DPoint.java:847)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.doTestRandom(TestGeo3DPoint.java:548)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig(TestGeo3DPoint.java:480)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 4) Thread[id=1, name=main, state=WAITING, group=main]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Thread.join(Thread.java:1252)
[junit4] 2> at java.lang.Thread.join(Thread.java:1326)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:636)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:493)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:251)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:368)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:13)
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestGeo3DPoint -Dtests.method=testRandomBig -Dtests.seed=88311BF75969D14 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=sk-SK -Dtests.timezone=Europe/Saratov -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 7161s J1 | TestGeo3DPoint.testRandomBig <<<
[junit4] > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
[junit4] > at __randomizedtesting.SeedInfo.seed([88311BF75969D14]:0)
[junit4] 2> sep 14, 2017 2:13:43 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> sep 14, 2017 2:14:03 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.lucene.spatial3d.TestGeo3DPoint:
[junit4] 2> 1) Thread[id=12, name=TEST-TestGeo3DPoint.testRandomBig-seed#[88311BF75969D14], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at org.apache.lucene.spatial3d.geom.Plane.findIntersections(Plane.java:650)
[junit4] 2> at org.apache.lucene.spatial3d.geom.StandardXYZSolid.<init>(StandardXYZSolid.java:189)
[junit4] 2> at org.apache.lucene.spatial3d.geom.XYZSolidFactory.makeXYZSolid(XYZSolidFactory.java:64)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.verify(TestGeo3DPoint.java:847)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.doTestRandom(TestGeo3DPoint.java:548)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig(TestGeo3DPoint.java:480)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> sep 14, 2017 2:14:03 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=12, name=TEST-TestGeo3DPoint.testRandomBig-seed#[88311BF75969D14], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/build/spatial3d/test/J1/temp/lucene.spatial3d.TestGeo3DPoint_88311BF75969D14-001
[junit4] 2> Sep 13, 2017 10:14:03 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: All leaked threads terminated.
[junit4] 2> NOTE: test params are: codec=Lucene70, sim=RandomSimilarity(queryNorm=true): {}, locale=sk-SK, timezone=Europe/Saratov
[junit4] 2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=219736224,total=448266240
[junit4] 2> NOTE: All tests run in this JVM: [TestGeo3DPoint]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestGe
[...truncated too long message...]
6) Thread[id=3463, name=zkCallback-1404-thread-4, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 7) Thread[id=2980, name=Connection evictor, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 8) Thread[id=4089, name=zkCallback-1404-thread-5, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 9) Thread[id=3466, name=zkCallback-1404-thread-4-EventThread, state=WAITING, group=TGRP-StreamExpressionTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)
[junit4] > 10) Thread[id=3457, name=zkCallback-1404-thread-2, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([C4BB7318CC98FC59]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=3458, name=zkCallback-1404-thread-3, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 2) Thread[id=2983, name=zkCallback-1404-thread-1, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 3) Thread[id=3465, name=zkCallback-1404-thread-4-SendThread(127.0.0.1:45246), state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)
[junit4] > 4) Thread[id=3463, name=zkCallback-1404-thread-4, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 5) Thread[id=4089, name=zkCallback-1404-thread-5, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 6) Thread[id=3457, name=zkCallback-1404-thread-2, state=TIMED_WAITING, group=TGRP-StreamExpressionTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([C4BB7318CC98FC59]:0)
[junit4] Completed [124/145 (1!)] on J2 in 372.46s, 109 tests, 1 failure, 2 errors, 9 skipped <<< FAILURES!
[...truncated 45383 lines...]
[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 44 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/44/
9 tests failed.
FAILED: org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([C52642C81A5F622D]:0)
FAILED: junit.framework.TestSuite.org.apache.lucene.spatial3d.TestGeo3DPoint
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([C52642C81A5F622D]:0)
FAILED: org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest.test
Error Message:
The Monkey ran for over 45 seconds and no jetties were stopped - this is worth investigating!
Stack Trace:
java.lang.AssertionError: The Monkey ran for over 45 seconds and no jetties were stopped - this is worth investigating!
at __randomizedtesting.SeedInfo.seed([CEDA8C94DD855290:468EB34E73793F68]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.ChaosMonkey.stopTheMonkey(ChaosMonkey.java:587)
at org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest.test(ChaosMonkeySafeLeaderWithPullReplicasTest.java:174)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest
Error Message:
7 threads leaked from SUITE scope at org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest: 1) Thread[id=50028, name=zkCallback-3031-thread-3, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=50000, name=StoppableCommitThread-EventThread, state=WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501) 3) Thread[id=50001, name=zkCallback-3031-thread-1, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=50027, name=zkCallback-3031-thread-2, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=50029, name=zkCallback-3031-thread-4, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 6) Thread[id=49976, name=StoppableCommitThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at java.lang.Thread.sleep(Native Method) at org.apache.solr.cloud.StoppableCommitThread.run(StoppableCommitThread.java:55) 7) Thread[id=49999, name=StoppableCommitThread-SendThread(127.0.0.1:43067), state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 7 threads leaked from SUITE scope at org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest:
1) Thread[id=50028, name=zkCallback-3031-thread-3, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=50000, name=StoppableCommitThread-EventThread, state=WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)
3) Thread[id=50001, name=zkCallback-3031-thread-1, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
4) Thread[id=50027, name=zkCallback-3031-thread-2, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
5) Thread[id=50029, name=zkCallback-3031-thread-4, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
6) Thread[id=49976, name=StoppableCommitThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.solr.cloud.StoppableCommitThread.run(StoppableCommitThread.java:55)
7) Thread[id=49999, name=StoppableCommitThread-SendThread(127.0.0.1:43067), state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)
at __randomizedtesting.SeedInfo.seed([CEDA8C94DD855290]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=50028, name=zkCallback-3031-thread-3, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=50001, name=zkCallback-3031-thread-1, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=50027, name=zkCallback-3031-thread-2, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=50029, name=zkCallback-3031-thread-4, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=49999, name=StoppableCommitThread-SendThread(127.0.0.1:43067), state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1051)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=50028, name=zkCallback-3031-thread-3, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=50001, name=zkCallback-3031-thread-1, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=50027, name=zkCallback-3031-thread-2, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
4) Thread[id=50029, name=zkCallback-3031-thread-4, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
5) Thread[id=49999, name=StoppableCommitThread-SendThread(127.0.0.1:43067), state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1051)
at __randomizedtesting.SeedInfo.seed([CEDA8C94DD855290]:0)
FAILED: org.apache.solr.cloud.LeaderElectionContextKeyTest.test
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([CEDA8C94DD855290:468EB34E73793F68]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.cloud.LeaderElectionContextKeyTest.test(LeaderElectionContextKeyTest.java:88)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload
Error Message:
expected:<[{indexVersion=1505194193090,generation=2,filelist=[_0.cfe, _0.cfs, _0.si, _1.cfe, _1.cfs, _1.si, _2.cfe, _2.cfs, _2.si, _3.cfe, _3.cfs, _3.si, _4.cfe, _4.cfs, _4.si, _5.cfe, _5.cfs, _5.si, segments_2]}]> but was:<[{indexVersion=1505194193090,generation=2,filelist=[_0.cfe, _0.cfs, _0.si, _1.cfe, _1.cfs, _1.si, _2.cfe, _2.cfs, _2.si, _3.cfe, _3.cfs, _3.si, _4.cfe, _4.cfs, _4.si, _5.cfe, _5.cfs, _5.si, segments_2]}, {indexVersion=1505194193090,generation=3,filelist=[_6.fdt, _6.fdx, _6.fnm, _6.nvd, _6.nvm, _6.si, _6_Lucene50_0.doc, _6_Lucene50_0.tim, _6_Lucene50_0.tip, segments_3]}]>
Stack Trace:
java.lang.AssertionError: expected:<[{indexVersion=1505194193090,generation=2,filelist=[_0.cfe, _0.cfs, _0.si, _1.cfe, _1.cfs, _1.si, _2.cfe, _2.cfs, _2.si, _3.cfe, _3.cfs, _3.si, _4.cfe, _4.cfs, _4.si, _5.cfe, _5.cfs, _5.si, segments_2]}]> but was:<[{indexVersion=1505194193090,generation=2,filelist=[_0.cfe, _0.cfs, _0.si, _1.cfe, _1.cfs, _1.si, _2.cfe, _2.cfs, _2.si, _3.cfe, _3.cfs, _3.si, _4.cfe, _4.cfs, _4.si, _5.cfe, _5.cfs, _5.si, segments_2]}, {indexVersion=1505194193090,generation=3,filelist=[_6.fdt, _6.fdx, _6.fnm, _6.nvd, _6.nvm, _6.si, _6_Lucene50_0.doc, _6_Lucene50_0.tim, _6_Lucene50_0.tip, segments_3]}]>
at __randomizedtesting.SeedInfo.seed([CEDA8C94DD855290:EB0D97A4ADCD5C93]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:147)
at org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload(TestReplicationHandler.java:1277)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.client.solrj.io.stream.JDBCStreamTest
Error Message:
Could not load collection from ZK: jdbc
Stack Trace:
org.apache.solr.common.SolrException: Could not load collection from ZK: jdbc
at __randomizedtesting.SeedInfo.seed([EDD75C170C66AF2A]:0)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1115)
at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:648)
at org.apache.solr.common.cloud.ClusterState.getCollectionOrNull(ClusterState.java:128)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:154)
at org.apache.solr.client.solrj.io.stream.JDBCStreamTest.setupCluster(JDBCStreamTest.java:77)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:874)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/jdbc/state.json
at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1212)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:357)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:354)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:354)
at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1127)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1113)
... 28 more
FAILED: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test
Error Message:
expected:<2> but was:<4>
Stack Trace:
java.lang.AssertionError: expected:<2> but was:<4>
at __randomizedtesting.SeedInfo.seed([CEDA8C94DD855290:468EB34E73793F68]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.assertSliceAndReplicaCount(SharedFSAutoReplicaFailoverTest.java:403)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.testBasics(SharedFSAutoReplicaFailoverTest.java:248)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test(SharedFSAutoReplicaFailoverTest.java:136)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 10544 lines...]
[junit4] Suite: org.apache.lucene.spatial3d.TestGeo3DPoint
[junit4] 2> Sep 12, 2017 11:05:10 AM com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
[junit4] 2> WARNING: Suite execution timed out: org.apache.lucene.spatial3d.TestGeo3DPoint
[junit4] 2> 1) Thread[id=9, name=JUnit4-serializer-daemon, state=TIMED_WAITING, group=main]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:50)
[junit4] 2> 2) Thread[id=14, name=TEST-TestGeo3DPoint.testRandomBig-seed#[C52642C81A5F622D], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at org.apache.lucene.spatial3d.geom.Plane.addPoint(Plane.java:1953)
[junit4] 2> at org.apache.lucene.spatial3d.geom.Plane.recordBounds(Plane.java:1418)
[junit4] 2> at org.apache.lucene.spatial3d.geom.XYZBounds.addPlane(XYZBounds.java:161)
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoConvexPolygon.getBounds(GeoConvexPolygon.java:484)
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoBaseCompositeShape.getBounds(GeoBaseCompositeShape.java:131)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.verify(TestGeo3DPoint.java:846)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.doTestRandom(TestGeo3DPoint.java:548)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig(TestGeo3DPoint.java:480)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 3) Thread[id=1, name=main, state=WAITING, group=main]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Thread.join(Thread.java:1252)
[junit4] 2> at java.lang.Thread.join(Thread.java:1326)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:636)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:493)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:251)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:368)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:13)
[junit4] 2> 4) Thread[id=13, name=SUITE-TestGeo3DPoint-seed#[C52642C81A5F622D], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at java.lang.Thread.getStackTrace(Thread.java:1559)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:696)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:693)
[junit4] 2> at java.security.AccessController.doPrivileged(Native Method)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getStackTrace(ThreadLeakControl.java:693)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getThreadsWithTraces(ThreadLeakControl.java:709)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:689)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:65)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:415)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:705)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:139)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:626)
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestGeo3DPoint -Dtests.method=testRandomBig -Dtests.seed=C52642C81A5F622D -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=de-AT -Dtests.timezone=Antarctica/Vostok -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 6833s J1 | TestGeo3DPoint.testRandomBig <<<
[junit4] > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
[junit4] > at __randomizedtesting.SeedInfo.seed([C52642C81A5F622D]:0)
[junit4] 2> Sep 12, 2017 11:05:10 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> Sep 12, 2017 11:05:30 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.lucene.spatial3d.TestGeo3DPoint:
[junit4] 2> 1) Thread[id=14, name=TEST-TestGeo3DPoint.testRandomBig-seed#[C52642C81A5F622D], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at org.apache.lucene.spatial3d.geom.Plane.findIntersections(Plane.java:650)
[junit4] 2> at org.apache.lucene.spatial3d.geom.StandardXYZSolid.<init>(StandardXYZSolid.java:179)
[junit4] 2> at org.apache.lucene.spatial3d.geom.XYZSolidFactory.makeXYZSolid(XYZSolidFactory.java:64)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.verify(TestGeo3DPoint.java:847)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.doTestRandom(TestGeo3DPoint.java:548)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig(TestGeo3DPoint.java:480)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Sep 12, 2017 11:05:30 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=14, name=TEST-TestGeo3DPoint.testRandomBig-seed#[C52642C81A5F622D], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/build/spatial3d/test/J1/temp/lucene.spatial3d.TestGeo3DPoint_C52642C81A5F622D-001
[junit4] 2> Sep 12, 2017 5:05:30 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: All leaked threads terminated.
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {id=PostingsFormat(name=LuceneVarGapFixedInterval)}, docValues:{id=DocValuesFormat(name=Asserting), point=DocValuesFormat(name=Direct)}, maxPointsInLeafNode=893, maxMBSortInHeap=7.88076471305585, sim=RandomSimilarity(queryNorm=false): {}, locale=de-AT, timezone=Antarctica/Vostok
[junit4] 2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=78215096,total=434110464
[junit4] 2> NOTE: All tests run in this JVM: [GeoCircleTest, TestGeo3DPoint]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestGeo3DPoint -Dtests.seed=C52642C81A5F622D -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=de-AT -Dtests.timezone=Antarctica/Vostok -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J1 | TestGeo3DPoint (suite) <<<
[junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4] > at __randomizedtesting.SeedInfo.seed([C52642C81A5F622D]:0)
[junit4] Completed [15/15 (1!)] on J1 in 7220.33s, 12 tests, 2 errors <<< FAILURES!
[...truncated 1964 lines...]
[junit4] Suite: org.apache.solr.handler.TestReplicationHandler
[junit4] 2> 626563 INFO (SUITE-TestReplicationHandler-seed#[CEDA8C94DD855290]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CEDA8C94DD855290-001/init-core-data-001
[junit4] 2> 626563 INFO (SUITE-TestReplicationHandler-seed#[CEDA8C94DD855290]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 626565 INFO (SUITE-TestReplicationHandler-seed#[CEDA8C94DD855290]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=None)
[junit4] 2> 626577 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testRateLimitedReplication
[junit4] 2> 626577 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CEDA8C94DD855290-001/solr-instance-001/collection1
[junit4] 2> 626580 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 626639 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2aeb645f{/solr,null,AVAILABLE}
[junit4] 2> 626641 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@31bc4b8b{HTTP/1.1,[http/1.1]}{127.0.0.1:35026}
[junit4] 2> 626641 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.e.j.s.Server Started @685771ms
[junit4] 2> 626641 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CEDA8C94DD855290-001/solr-instance-001/collection1/data, hostContext=/solr, hostPort=35026}
[junit4] 2> 626641 ERROR (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 626641 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
[junit4] 2> 626641 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 626641 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 626642 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-09-12T05:25:16.985Z
[junit4] 2> 626642 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CEDA8C94DD855290-001/solr-instance-001/solr.xml
[junit4] 2> 626647 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@45fbee32, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 626754 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@45fbee32
[junit4] 2> 626762 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@45fbee32
[junit4] 2> 626762 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@45fbee32
[junit4] 2> 626764 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CEDA8C94DD855290-001/solr-instance-001/.
[junit4] 2> 626765 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 626862 INFO (coreLoadExecutor-215-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 626886 INFO (coreLoadExecutor-215-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 626888 INFO (coreLoadExecutor-215-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
[junit4] 2> 626888 INFO (coreLoadExecutor-215-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CEDA8C94DD855290-001/solr-instance-001/./collection1, trusted=true
[junit4] 2> 626895 INFO (coreLoadExecutor-215-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@45fbee32
[junit4] 2> 626895 INFO (coreLoadExecutor-215-thread-1) [ x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 626895 INFO (coreLoadExecutor-215-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CEDA8C94DD855290-001/solr-instance-001/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CEDA8C94DD855290-001/solr-instance-001/./collection1/data/]
[junit4] 2> 626897 INFO (coreLoadExecutor-215-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=657264804, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 626998 INFO (coreLoadExecutor-215-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 626998 INFO (coreLoadExecutor-215-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 627006 INFO (coreLoadExecutor-215-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=6, maxMergeAtOnceExplicit=4, maxMergedSegmentMB=69.0673828125, floorSegmentMB=2.037109375, forceMergeDeletesPctAllowed=1.5590187699245595, segmentsPerTier=33.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 627007 INFO (coreLoadExecutor-215-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3c9de413[collection1] main]
[junit4] 2> 627007 INFO (coreLoadExecutor-215-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CEDA8C94DD855290-001/solr-instance-001/collection1/conf
[junit4] 2> 627008 INFO (coreLoadExecutor-215-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema.xml,xslt/dummy.xsl
[junit4] 2> 627008 INFO (coreLoadExecutor-215-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 627009 INFO (searcherExecutor-216-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@3c9de413[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 627375 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CEDA8C94DD855290-001/solr-instance-002/collection1
[junit4] 2> 627377 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 627391 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@37ba8d2e{/solr,null,AVAILABLE}
[junit4] 2> 627391 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@554a0a94{HTTP/1.1,[http/1.1]}{127.0.0.1:39223}
[junit4] 2> 627391 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.e.j.s.Server Started @686522ms
[junit4] 2> 627391 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CEDA8C94DD855290-001/solr-instance-002/collection1/data, hostContext=/solr, hostPort=39223}
[junit4] 2> 627392 ERROR (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 627392 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
[junit4] 2> 627392 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 627392 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 627392 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-09-12T05:25:17.735Z
[junit4] 2> 627392 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CEDA8C94DD855290-001/solr-instance-002/solr.xml
[junit4] 2> 627397 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@45fbee32, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 627435 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@45fbee32
[junit4] 2> 627445 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@45fbee32
[junit4] 2> 627445 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@45fbee32
[junit4] 2> 627456 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CEDA8C94DD855290-001/solr-instance-002/.
[junit4] 2> 627456 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 627505 INFO (coreLoadExecutor-225-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 627525 INFO (coreLoadExecutor-225-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 627526 INFO (coreLoadExecutor-225-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
[junit4] 2> 627526 INFO (coreLoadExecutor-225-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CEDA8C94DD855290-001/solr-instance-002/./collection1, trusted=true
[junit4] 2> 627527 INFO (coreLoadExecutor-225-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@45fbee32
[junit4] 2> 627527 INFO (coreLoadExecutor-225-thread-1) [ x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 627527 INFO (coreLoadExecutor-225-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CEDA8C94DD855290-001/solr-instance-002/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CEDA8C94DD855290-001/solr-instance-002/./collection1/data/]
[junit4] 2> 627535 INFO (coreLoadExecutor-225-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=657264804, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 627630 INFO (coreLoadExecutor-225-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 627630 INFO (coreLoadExecutor-225-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 627631 INFO (coreLoadExecutor-225-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=6, maxMergeAtOnceExplicit=4, maxMergedSegmentMB=69.0673828125, floorSegmentMB=2.037109375, forceMergeDeletesPctAllowed=1.5590187699245595, segmentsPerTier=33.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 627632 INFO (coreLoadExecutor-225-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@dd16852[collection1] main]
[junit4] 2> 627632 INFO (coreLoadExecutor-225-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CEDA8C94DD855290-001/solr-instance-002/collection1/conf
[junit4] 2> 627642 INFO (coreLoadExecutor-225-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 1000ms
[junit4] 2> 627649 INFO (searcherExecutor-226-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@dd16852[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 627657 INFO (qtp682652873-1610) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 627657 INFO (indexFetcher-230-thread-1) [ x:collection1] o.a.s.h.IndexFetcher Master's generation: 1
[junit4] 2> 627657 INFO (indexFetcher-230-thread-1) [ x:collection1] o.a.s.h.IndexFetcher Master's version: 0
[junit4] 2> 627657 INFO (indexFetcher-230-thread-1) [ x:collection1] o.a.s.h.IndexFetcher Slave's generation: 1
[junit4] 2> 627657 INFO (indexFetcher-230-thread-1) [ x:collection1] o.a.s.h.IndexFetcher Slave's version: 0
[junit4] 2> 628024 INFO (qtp682652873-1612) [ x:collection1] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
[junit4] 2> 628025 INFO (qtp682652873-1612) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{deleteByQuery=*:*} 0 1
[junit4] 2> 628039 INFO (qtp1068252288-1627) [ x:collection1] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
[junit4] 2> 628039 INFO (qtp1068252288-1627) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{deleteByQuery=*:*} 0 0
[junit4] 2> 628040 INFO (qtp682652873-1613) [ x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 628040 INFO (qtp682652873-1613) [ x:collection1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@120fe8c2 commitCommandVersion:0
[junit4] 2> 628041 INFO (qtp682652873-1613) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5df51ec8[collection1] main]
[junit4] 2> 628041 INFO (qtp682652873-1613) [ x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 628042 INFO (searcherExecutor-216-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5df51ec8[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 628042 INFO (qtp682652873-1613) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 2
[junit4] 2> 628043 INFO (qtp1068252288-1628) [ x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 628043 INFO (qtp1068252288-1628) [ x:collection1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@13ffb9cf commitCommandVersion:0
[junit4] 2> 628044 INFO (qtp1068252288-1628) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5c055ffb[collection1] main]
[junit4] 2> 628044 INFO (qtp1068252288-1628) [ x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 628045 INFO (searcherExecutor-226-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5c055ffb[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 628045 INFO (qtp1068252288-1628) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 2
[junit4] 2> 628046 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@31bc4b8b{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 628046 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=550576321
[junit4] 2> 628046 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 628046 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@ff8bf0a: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@77684a2b
[junit4] 2> 628050 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 628050 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@8d77036: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@53f871d7
[junit4] 2> 628052 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 628052 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@ee08ac6: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@3cda2958
[junit4] 2> 628072 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 628086 INFO (coreCloseExecutor-231-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@5e431387
[junit4] 2> 628086 INFO (coreCloseExecutor-231-thread-1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=1581454215
[junit4] 2> 628086 INFO (coreCloseExecutor-231-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@b2399: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@227a96c8
[junit4] 2> 628105 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2aeb645f{/solr,null,UNAVAILABLE}
[junit4] 2> 628107 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@554a0a94{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 628108 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=558576568
[junit4] 2> 628116 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 628116 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@260e537e: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@5f865bc
[junit4] 2> 628120 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 628120 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5e9d2c07: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@53f871d7
[junit4] 2> 628124 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 628124 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1e874ffc: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@3cda2958
[junit4] 2> 628124 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 628125 INFO (coreCloseExecutor-233-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@18384f9e
[junit4] 2> 628125 INFO (coreCloseExecutor-233-thread-1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=406343582
[junit4] 2> 628125 INFO (coreCloseExecutor-233-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5a31ce6d: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@27e281e1
[junit4] 2> 628156 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@37ba8d2e{/solr,null,UNAVAILABLE}
[junit4] 2> 628157 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 628161 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@576e3839{/solr,null,AVAILABLE}
[junit4] 2> 628163 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@63b92c8d{HTTP/1.1,[http/1.1]}{127.0.0.1:32938}
[junit4] 2> 628163 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.e.j.s.Server Started @687293ms
[junit4] 2> 628163 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CEDA8C94DD855290-001/solr-instance-001/collection1/data, hostContext=/solr, hostPort=32938}
[junit4] 2> 628164 ERROR (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 628164 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
[junit4] 2> 628164 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 628164 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 628164 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-09-12T05:25:18.507Z
[junit4] 2> 628164 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CEDA8C94DD855290-001/solr-instance-001/solr.xml
[junit4] 2> 628169 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@45fbee32, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 628224 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@45fbee32
[junit4] 2> 628232 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@45fbee32
[junit4] 2> 628232 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@45fbee32
[junit4] 2> 628234 INFO (TEST-TestReplicationHandler.testRateLimitedReplication-seed#[CEDA8C94DD855290]) [ ] o.a.s.c.
[...truncated too long message...]
lback-29-thread-1-processing-n:127.0.0.1:38969_solr) [n:127.0.0.1:38969_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:38969_solr
[junit4] 2> 102096 INFO (jetty-closer-8-thread-3) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 102096 INFO (zkCallback-28-thread-1-processing-n:127.0.0.1:45363_solr) [n:127.0.0.1:45363_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 102096 INFO (zkCallback-29-thread-2-processing-n:127.0.0.1:38969_solr) [n:127.0.0.1:38969_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 102096 INFO (jetty-closer-8-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 102096 INFO (jetty-closer-8-thread-1) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 102362 INFO (coreCloseExecutor-53-thread-1) [n:127.0.0.1:45363_solr c:jdbc s:shard2 r:core_node4 x:jdbc_shard2_replica_n2] o.a.s.c.SolrCore [jdbc_shard2_replica_n2] CLOSING SolrCore org.apache.solr.core.SolrCore@30a6d815
[junit4] 2> 102362 INFO (coreCloseExecutor-53-thread-1) [n:127.0.0.1:45363_solr c:jdbc s:shard2 r:core_node4 x:jdbc_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.jdbc.shard2.replica_n2, tag=816240661
[junit4] 2> 102362 INFO (coreCloseExecutor-53-thread-1) [n:127.0.0.1:45363_solr c:jdbc s:shard2 r:core_node4 x:jdbc_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@127495fe: rootName = solr_45363, domain = solr.core.jdbc.shard2.replica_n2, service url = null, agent id = null] for registry solr.core.jdbc.shard2.replica_n2 / com.codahale.metrics.MetricRegistry@56048cbf
[junit4] 2> 102472 INFO (coreCloseExecutor-51-thread-1) [n:127.0.0.1:38969_solr c:jdbc s:shard1 r:core_node3 x:jdbc_shard1_replica_n1] o.a.s.c.SolrCore [jdbc_shard1_replica_n1] CLOSING SolrCore org.apache.solr.core.SolrCore@41e5cb
[junit4] 2> 102473 INFO (coreCloseExecutor-51-thread-1) [n:127.0.0.1:38969_solr c:jdbc s:shard1 r:core_node3 x:jdbc_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.jdbc.shard1.replica_n1, tag=4318667
[junit4] 2> 102473 INFO (coreCloseExecutor-51-thread-1) [n:127.0.0.1:38969_solr c:jdbc s:shard1 r:core_node3 x:jdbc_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6a9391ec: rootName = solr_38969, domain = solr.core.jdbc.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.jdbc.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@9e39f6a
[junit4] 2> 102562 INFO (coreCloseExecutor-53-thread-1) [n:127.0.0.1:45363_solr c:jdbc s:shard2 r:core_node4 x:jdbc_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.jdbc.shard2.leader, tag=816240661
[junit4] 2> 102699 INFO (zkCallback-29-thread-2-processing-n:127.0.0.1:38969_solr) [n:127.0.0.1:38969_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 102719 INFO (coreCloseExecutor-51-thread-1) [n:127.0.0.1:38969_solr c:jdbc s:shard1 r:core_node3 x:jdbc_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.jdbc.shard1.leader, tag=4318667
[junit4] 2> 102739 INFO (zkCallback-28-thread-1-processing-n:127.0.0.1:45363_solr) [n:127.0.0.1:45363_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 102910 INFO (jetty-closer-8-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@4fbe3807{/solr,null,UNAVAILABLE}
[junit4] 2> 103012 WARN (zkCallback-28-thread-2-processing-n:127.0.0.1:45363_solr) [n:127.0.0.1:45363_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 103013 INFO (jetty-closer-8-thread-3) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@123a0d37{/solr,null,UNAVAILABLE}
[junit4] 2> 103479 WARN (zkCallback-26-thread-2-processing-n:127.0.0.1:38144_solr) [n:127.0.0.1:38144_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 103491 INFO (jetty-closer-8-thread-4) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@3b73141c{/solr,null,UNAVAILABLE}
[junit4] 2> 104513 WARN (zkCallback-29-thread-1-processing-n:127.0.0.1:38969_solr) [n:127.0.0.1:38969_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 104513 INFO (jetty-closer-8-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@14da6925{/solr,null,UNAVAILABLE}
[junit4] 2> 104547 ERROR (SUITE-JDBCStreamTest-seed#[EDD75C170C66AF2A]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 104585 INFO (SUITE-JDBCStreamTest-seed#[EDD75C170C66AF2A]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:44835 44835
[junit4] 2> 104685 INFO (Thread-20) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:44835 44835
[junit4] 2> 104686 WARN (Thread-20) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 6 /solr/aliases.json
[junit4] 2> 6 /solr/clusterprops.json
[junit4] 2> 4 /solr/security.json
[junit4] 2> 2 /solr/configs/conf
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 14 /solr/collections/jdbc/state.json
[junit4] 2> 6 /solr/clusterstate.json
[junit4] 2> 2 /solr/overseer_elect/election/98644889381961740-127.0.0.1:38969_solr-n_0000000001
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 6 /solr/live_nodes
[junit4] 2> 6 /solr/collections
[junit4] 2>
[junit4] 2> 104686 INFO (SUITE-JDBCStreamTest-seed#[EDD75C170C66AF2A]-worker) [ ] o.a.s.SolrTestCaseJ4 ------------------------------------------------------- Done waiting for tracked resources to be released
[junit4] 2> Sep 12, 2017 7:33:10 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=519, maxMBSortInHeap=5.645504956179867, sim=RandomSimilarity(queryNorm=true): {}, locale=cs, timezone=Pacific/Johnston
[junit4] 2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=234239304,total=318767104
[junit4] 2> NOTE: All tests run in this JVM: [OrEvaluatorTest, SolrZkClientTest, AppendEvaluatorTest, SubtractEvaluatorTest, LargeVolumeBinaryJettyTest, StreamExpressionToExpessionTest, JDBCStreamTest]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=JDBCStreamTest -Dtests.seed=EDD75C170C66AF2A -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=cs -Dtests.timezone=Pacific/Johnston -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J0 | JDBCStreamTest (suite) <<<
[junit4] > Throwable #1: org.apache.solr.common.SolrException: Could not load collection from ZK: jdbc
[junit4] > at __randomizedtesting.SeedInfo.seed([EDD75C170C66AF2A]:0)
[junit4] > at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1115)
[junit4] > at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:648)
[junit4] > at org.apache.solr.common.cloud.ClusterState.getCollectionOrNull(ClusterState.java:128)
[junit4] > at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:154)
[junit4] > at org.apache.solr.client.solrj.io.stream.JDBCStreamTest.setupCluster(JDBCStreamTest.java:77)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/jdbc/state.json
[junit4] > at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] > at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] > at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1212)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:357)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:354)
[junit4] > at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:354)
[junit4] > at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1127)
[junit4] > at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1113)
[junit4] > ... 28 more
[junit4] Completed [23/145 (1!)] on J0 in 36.10s, 0 tests, 1 error <<< FAILURES!
[...truncated 45685 lines...]
[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 43 - Still
unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/43/
8 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsBasicDistributedZkTest
Error Message:
ObjectTracker found 1 object(s) that were not released!!! [NRTCachingDirectory] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348) at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:349) at org.apache.solr.core.SolrCore.cleanupOldIndexDirectories(SolrCore.java:3011) at org.apache.solr.core.SolrCore.close(SolrCore.java:1584) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:980) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:843) at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:998) at org.apache.solr.core.CoreContainer.lambda$load$13(CoreContainer.java:637) at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
java.lang.AssertionError: ObjectTracker found 1 object(s) that were not released!!! [NRTCachingDirectory]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:349)
at org.apache.solr.core.SolrCore.cleanupOldIndexDirectories(SolrCore.java:3011)
at org.apache.solr.core.SolrCore.close(SolrCore.java:1584)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:980)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:843)
at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:998)
at org.apache.solr.core.CoreContainer.lambda$load$13(CoreContainer.java:637)
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([197712C30A2173D9]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:294)
at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:897)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.hdfs.HdfsRecoveryZkTest.test
Error Message:
Stack Trace:
java.util.concurrent.TimeoutException
at __randomizedtesting.SeedInfo.seed([197712C30A2173D9:91232D19A4DD1E21]:0)
at org.apache.solr.common.cloud.ZkStateReader.waitForState(ZkStateReader.java:1268)
at org.apache.solr.client.solrj.impl.CloudSolrClient.waitForState(CloudSolrClient.java:438)
at org.apache.solr.cloud.RecoveryZkTest.test(RecoveryZkTest.java:122)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.handler.TestSolrConfigHandlerConcurrent.test
Error Message:
KeeperErrorCode = Session expired for /clusterstate.json
Stack Trace:
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /clusterstate.json
at __randomizedtesting.SeedInfo.seed([197712C30A2173D9:91232D19A4DD1E21]:0)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1212)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:357)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:354)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:354)
at org.apache.solr.common.cloud.ZkStateReader.refreshLegacyClusterState(ZkStateReader.java:542)
at org.apache.solr.common.cloud.ZkStateReader.forceUpdateCollection(ZkStateReader.java:310)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.updateMappingsFromZk(AbstractFullDistribZkTestBase.java:674)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.updateMappingsFromZk(AbstractFullDistribZkTestBase.java:669)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createJettys(AbstractFullDistribZkTestBase.java:464)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:334)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:991)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.TestCloudRecovery.corruptedLogTest
Error Message:
Timeout waiting for all live and active
Stack Trace:
java.lang.AssertionError: Timeout waiting for all live and active
at __randomizedtesting.SeedInfo.seed([197712C30A2173D9:9A014D31DC587D78]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.TestCloudRecovery.corruptedLogTest(TestCloudRecovery.java:184)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.hdfs.HdfsBasicDistributedZk2Test.test
Error Message:
.response.numFound:36!=67
Stack Trace:
junit.framework.AssertionFailedError: .response.numFound:36!=67
at __randomizedtesting.SeedInfo.seed([197712C30A2173D9:91232D19A4DD1E21]:0)
at junit.framework.Assert.fail(Assert.java:50)
at org.apache.solr.BaseDistributedSearchTestCase.compareSolrResponses(BaseDistributedSearchTestCase.java:913)
at org.apache.solr.BaseDistributedSearchTestCase.compareResponses(BaseDistributedSearchTestCase.java:940)
at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:607)
at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:587)
at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:566)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkQueries(AbstractFullDistribZkTestBase.java:938)
at org.apache.solr.cloud.BasicDistributedZk2Test.test(BasicDistributedZk2Test.java:102)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.update.TestInPlaceUpdatesDistrib.test
Error Message:
This doc was supposed to have been deleted, but was: SolrDocument{id=0, title_s=title0, id_i=0, inplace_updatable_float=1.0, _version_=1578205196421431296, inplace_updatable_int_with_default=666, inplace_updatable_float_with_default=42.0}
Stack Trace:
java.lang.AssertionError: This doc was supposed to have been deleted, but was: SolrDocument{id=0, title_s=title0, id_i=0, inplace_updatable_float=1.0, _version_=1578205196421431296, inplace_updatable_int_with_default=666, inplace_updatable_float_with_default=42.0}
at __randomizedtesting.SeedInfo.seed([197712C30A2173D9:91232D19A4DD1E21]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at org.apache.solr.update.TestInPlaceUpdatesDistrib.reorderedDBQsSimpleTest(TestInPlaceUpdatesDistrib.java:247)
at org.apache.solr.update.TestInPlaceUpdatesDistrib.test(TestInPlaceUpdatesDistrib.java:151)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.HttpPartitionTest.test
Error Message:
Doc with id=994 not found in http://127.0.0.1:44848/sjvbq/c8n_1x2 due to: Path not found: /id; rsp={doc=null}
Stack Trace:
java.lang.AssertionError: Doc with id=994 not found in http://127.0.0.1:44848/sjvbq/c8n_1x2 due to: Path not found: /id; rsp={doc=null}
at __randomizedtesting.SeedInfo.seed([197712C30A2173D9:91232D19A4DD1E21]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.HttpPartitionTest.assertDocExists(HttpPartitionTest.java:605)
at org.apache.solr.cloud.HttpPartitionTest.assertDocsExistInAllReplicas(HttpPartitionTest.java:560)
at org.apache.solr.cloud.HttpPartitionTest.testRf2(HttpPartitionTest.java:381)
at org.apache.solr.cloud.HttpPartitionTest.test(HttpPartitionTest.java:132)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test
Error Message:
expected:<2> but was:<3>
Stack Trace:
java.lang.AssertionError: expected:<2> but was:<3>
at __randomizedtesting.SeedInfo.seed([197712C30A2173D9:91232D19A4DD1E21]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.assertSliceAndReplicaCount(SharedFSAutoReplicaFailoverTest.java:403)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.testBasics(SharedFSAutoReplicaFailoverTest.java:248)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test(SharedFSAutoReplicaFailoverTest.java:136)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12312 lines...]
[junit4] Suite: org.apache.solr.cloud.hdfs.HdfsBasicDistributedZk2Test
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_197712C30A2173D9-001/init-core-data-001
[junit4] 2> 1004251 WARN (SUITE-HdfsBasicDistributedZk2Test-seed#[197712C30A2173D9]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=5 numCloses=5
[junit4] 2> 1004251 INFO (SUITE-HdfsBasicDistributedZk2Test-seed#[197712C30A2173D9]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 1004252 INFO (SUITE-HdfsBasicDistributedZk2Test-seed#[197712C30A2173D9]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
[junit4] 2> 1004252 INFO (SUITE-HdfsBasicDistributedZk2Test-seed#[197712C30A2173D9]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 1004253 INFO (SUITE-HdfsBasicDistributedZk2Test-seed#[197712C30A2173D9]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /q_zfv/sk
[junit4] 2> 1006731 WARN (SUITE-HdfsBasicDistributedZk2Test-seed#[197712C30A2173D9]-worker) [ ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 1015206 WARN (SUITE-HdfsBasicDistributedZk2Test-seed#[197712C30A2173D9]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 1018479 INFO (SUITE-HdfsBasicDistributedZk2Test-seed#[197712C30A2173D9]-worker) [ ] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
[junit4] 2> 1018577 WARN (SUITE-HdfsBasicDistributedZk2Test-seed#[197712C30A2173D9]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 1021441 INFO (SUITE-HdfsBasicDistributedZk2Test-seed#[197712C30A2173D9]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 1021555 INFO (SUITE-HdfsBasicDistributedZk2Test-seed#[197712C30A2173D9]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_32972_hdfs____.c759a9/webapp
[junit4] 2> 1029006 INFO (SUITE-HdfsBasicDistributedZk2Test-seed#[197712C30A2173D9]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:32972
[junit4] 2> 1039029 WARN (SUITE-HdfsBasicDistributedZk2Test-seed#[197712C30A2173D9]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 1039050 INFO (SUITE-HdfsBasicDistributedZk2Test-seed#[197712C30A2173D9]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 1039480 INFO (SUITE-HdfsBasicDistributedZk2Test-seed#[197712C30A2173D9]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_47655_datanode____.xguxly/webapp
[junit4] 2> 1043265 INFO (SUITE-HdfsBasicDistributedZk2Test-seed#[197712C30A2173D9]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:47655
[junit4] 2> 1049706 WARN (SUITE-HdfsBasicDistributedZk2Test-seed#[197712C30A2173D9]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 1049709 INFO (SUITE-HdfsBasicDistributedZk2Test-seed#[197712C30A2173D9]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 1049736 INFO (SUITE-HdfsBasicDistributedZk2Test-seed#[197712C30A2173D9]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_53836_datanode____de0gbo/webapp
[junit4] 2> 1055555 INFO (SUITE-HdfsBasicDistributedZk2Test-seed#[197712C30A2173D9]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:53836
[junit4] 2> 1066409 ERROR (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_197712C30A2173D9-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_197712C30A2173D9-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to localhost/127.0.0.1:59829) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 1066430 ERROR (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_197712C30A2173D9-001/tempDir-001/hdfsBaseDir/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_197712C30A2173D9-001/tempDir-001/hdfsBaseDir/data/data4/]] heartbeating to localhost/127.0.0.1:59829) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 1067250 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x12f5f33a42315b: from storage DS-2b14f5e8-10b1-485a-a814-0ba903708bb8 node DatanodeRegistration(127.0.0.1:50454, datanodeUuid=b25e0c00-ff54-4244-bdcc-53a34dcef2bf, infoPort=49606, infoSecurePort=0, ipcPort=55183, storageInfo=lv=-56;cid=testClusterID;nsid=1040292742;c=0), blocks: 0, hasStaleStorage: true, processing time: 18 msecs
[junit4] 2> 1067251 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x12f5f33a1fe139: from storage DS-97467601-ae00-4953-8489-c1ed5e29724c node DatanodeRegistration(127.0.0.1:43279, datanodeUuid=c84079ee-1866-423f-8bc5-12686584118d, infoPort=49455, infoSecurePort=0, ipcPort=49362, storageInfo=lv=-56;cid=testClusterID;nsid=1040292742;c=0), blocks: 0, hasStaleStorage: true, processing time: 1 msecs
[junit4] 2> 1067251 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x12f5f33a42315b: from storage DS-a2bc4141-8124-40cc-a438-bb70727e7eb4 node DatanodeRegistration(127.0.0.1:50454, datanodeUuid=b25e0c00-ff54-4244-bdcc-53a34dcef2bf, infoPort=49606, infoSecurePort=0, ipcPort=55183, storageInfo=lv=-56;cid=testClusterID;nsid=1040292742;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 1067254 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x12f5f33a1fe139: from storage DS-1449d293-540f-4d41-8863-801ade43ef85 node DatanodeRegistration(127.0.0.1:43279, datanodeUuid=c84079ee-1866-423f-8bc5-12686584118d, infoPort=49455, infoSecurePort=0, ipcPort=49362, storageInfo=lv=-56;cid=testClusterID;nsid=1040292742;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 1071522 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1071534 INFO (Thread-491) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1071534 INFO (Thread-491) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1071561 ERROR (Thread-491) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1071634 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.a.s.c.ZkTestServer start zk server on port:53174
[junit4] 2> 1071776 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 1071777 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 1071782 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1071787 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 1071787 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 1071788 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 1071789 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 1071802 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 1071810 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1071811 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 1071815 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 1071826 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use TLOG replicas unless explicitly asked otherwise
[junit4] 2> 1073191 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 1073194 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2ef7b2cf{/q_zfv/sk,null,AVAILABLE}
[junit4] 2> 1073194 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7ef633e6{HTTP/1.1,[http/1.1]}{127.0.0.1:52911}
[junit4] 2> 1073194 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.e.j.s.Server Started @1091901ms
[junit4] 2> 1073194 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:59829/hdfs__localhost_59829__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-7.x_checkout_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.HdfsBasicDistributedZk2Test_197712C30A2173D9-001_tempDir-002_control_data, replicaType=NRT, hostContext=/q_zfv/sk, hostPort=52911, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_197712C30A2173D9-001/control-001/cores}
[junit4] 2> 1073195 ERROR (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1073195 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
[junit4] 2> 1073195 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1073195 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 1073195 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-09-10T23:10:53.165Z
[junit4] 2> 1073224 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1073224 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_197712C30A2173D9-001/control-001/solr.xml
[junit4] 2> 1073245 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@244f7a6d, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 1073264 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53174/solr
[junit4] 2> 1073856 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [n:127.0.0.1:52911_q_zfv%2Fsk ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1073856 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [n:127.0.0.1:52911_q_zfv%2Fsk ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:52911_q_zfv%2Fsk
[junit4] 2> 1073860 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [n:127.0.0.1:52911_q_zfv%2Fsk ] o.a.s.c.Overseer Overseer (id=98637253935628292-127.0.0.1:52911_q_zfv%2Fsk-n_0000000000) starting
[junit4] 2> 1073977 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [n:127.0.0.1:52911_q_zfv%2Fsk ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52911_q_zfv%2Fsk
[junit4] 2> 1073983 INFO (zkCallback-172-thread-1-processing-n:127.0.0.1:52911_q_zfv%2Fsk) [n:127.0.0.1:52911_q_zfv%2Fsk ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1074759 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [n:127.0.0.1:52911_q_zfv%2Fsk ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@244f7a6d
[junit4] 2> 1074951 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [n:127.0.0.1:52911_q_zfv%2Fsk ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@244f7a6d
[junit4] 2> 1074952 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [n:127.0.0.1:52911_q_zfv%2Fsk ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@244f7a6d
[junit4] 2> 1074953 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [n:127.0.0.1:52911_q_zfv%2Fsk ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_197712C30A2173D9-001/control-001/cores
[junit4] 2> 1075551 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1075551 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:53174/solr ready
[junit4] 2> 1075587 INFO (qtp1479573790-2559) [n:127.0.0.1:52911_q_zfv%2Fsk ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:52911_q_zfv%252Fsk&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1075608 INFO (OverseerThreadFactory-339-thread-1-processing-n:127.0.0.1:52911_q_zfv%2Fsk) [n:127.0.0.1:52911_q_zfv%2Fsk ] o.a.s.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 1075738 INFO (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1075738 INFO (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 1075845 INFO (zkCallback-172-thread-1-processing-n:127.0.0.1:52911_q_zfv%2Fsk) [n:127.0.0.1:52911_q_zfv%2Fsk ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1076781 INFO (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 1077078 INFO (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
[junit4] 2> 1078284 INFO (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 1078350 INFO (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
[junit4] 2> 1078351 INFO (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@244f7a6d
[junit4] 2> 1080129 INFO (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:59829/solr_hdfs_home
[junit4] 2> 1080129 INFO (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 1080129 INFO (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1080129 INFO (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_197712C30A2173D9-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[hdfs://localhost:59829/solr_hdfs_home/control_collection/core_node2/data/]
[junit4] 2> 1080149 INFO (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:59829/solr_hdfs_home/control_collection/core_node2/data/snapshot_metadata
[junit4] 2> 1080210 INFO (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1080210 INFO (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 1080210 INFO (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 1085670 INFO (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1085678 INFO (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:59829/solr_hdfs_home/control_collection/core_node2/data
[junit4] 2> 1085828 INFO (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:59829/solr_hdfs_home/control_collection/core_node2/data/index
[junit4] 2> 1085860 INFO (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1085860 INFO (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 1085860 INFO (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 1087852 INFO (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1087853 INFO (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=39, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 1089202 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:50454 is added to blk_1073741825_1001{UCState=COMMITTED, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-1449d293-540f-4d41-8863-801ade43ef85:NORMAL:127.0.0.1:43279|RBW], ReplicaUC[[DISK]DS-2b14f5e8-10b1-485a-a814-0ba903708bb8:NORMAL:127.0.0.1:50454|RBW]]} size 72
[junit4] 2> 1089203 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:43279 is added to blk_1073741825_1001 size 72
[junit4] 2> 1089208 WARN (PacketResponder: BP-351018995-10.41.0.5-1505084993481:blk_1073741825_1001, type=LAST_IN_PIPELINE, downstreams=0:[]) [ ] o.a.h.h.s.d.DataNode Slow PacketResponder send ack to upstream took 657ms (threshold=300ms), PacketResponder: BP-351018995-10.41.0.5-1505084993481:blk_1073741825_1001, type=LAST_IN_PIPELINE, downstreams=0:[], replyAck=seqno: 1 reply: SUCCESS downstreamAckTimeNanos: 0 flag: 0
[junit4] 2> 1089646 WARN (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1090713 INFO (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 1090713 INFO (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1090713 INFO (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 1090779 INFO (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1090779 INFO (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1090784 INFO (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=3, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 1091176 INFO (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3ad71aa6[control_collection_shard1_replica_n1] main]
[junit4] 2> 1091182 INFO (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1091182 INFO (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1091185 INFO (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1091210 INFO (searcherExecutor-342-thread-1-processing-n:127.0.0.1:52911_q_zfv%2Fsk x:control_collection_shard1_replica_n1 s:shard1 c:control_collection) [n:127.0.0.1:52911_q_zfv%2Fsk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@3ad71aa6[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1091211 INFO (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1578196083598688256
[junit4] 2> 1091218 INFO (zkCallback-172-thread-1-processing-n:127.0.0.1:52911_q_zfv%2Fsk) [n:127.0.0.1:52911_q_zfv%2Fsk ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1091223 INFO (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1091223 INFO (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1091223 INFO (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:52911/q_zfv/sk/control_collection_shard1_replica_n1/
[junit4] 2> 1091224 INFO (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1091224 INFO (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:52911/q_zfv/sk/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 1091224 INFO (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 1091231 INFO (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:52911/q_zfv/sk/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 1091339 INFO (zkCallback-172-thread-1-processing-n:127.0.0.1:52911_q_zfv%2Fsk) [n:127.0.0.1:52911_q_zfv%2Fsk ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1091388 INFO (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1091390 INFO (qtp1479573790-2557) [n:127.0.0.1:52911_q_zfv%2Fsk c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=15652
[junit4] 2> 1091407 INFO (qtp1479573790-2559) [n:127.0.0.1:52911_q_zfv%2Fsk ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 1091501 INFO (zkCallback-172-thread-1-processing-n:127.0.0.1:52911_q_zfv%2Fsk) [n:127.0.0.1:52911_q_zfv%2Fsk ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1092122 INFO (OverseerCollectionConfigSetProcessor-98637253935628292-127.0.0.1:52911_q_zfv%2Fsk-n_0000000000) [n:127.0.0.1:52911_q_zfv%2Fsk ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1092461 INFO (qtp1479573790-2559) [n:127.0.0.1:52911_q_zfv%2Fsk ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:52911_q_zfv%252Fsk&wt=javabin&version=2} status=0 QTime=16873
[junit4] 2> 1092482 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1092483 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:53174/solr ready
[junit4] 2> 1092483 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 1092498 INFO (qtp1479573790-2560) [n:127.0.0.1:52911_q_zfv%2Fsk ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1092502 INFO (OverseerThreadFactory-339-thread-2-processing-n:127.0.0.1:52911_q_zfv%2Fsk) [n:127.0.0.1:52911_q_zfv%2Fsk ] o.a.s.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 1092503 WARN (OverseerThreadFactory-339-thread-2-processing-n:127.0.0.1:52911_q_zfv%2Fsk) [n:127.0.0.1:52911_q_zfv%2Fsk ] o.a.s.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
[junit4] 2> 1092621 INFO (qtp1479573790-2560) [n:127.0.0.1:52911_q_zfv%2Fsk ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 1092621 INFO (qtp1479573790-2560) [n:127.0.0.1:52911_q_zfv%2Fsk ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=1&wt=javabin&version=2} status=0 QTime=123
[junit4] 2> 1093765 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_197712C30A2173D9-001/shard-1-001 of type TLOG
[junit4] 2> 1093780 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 1093782 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1d33b4ea{/q_zfv/sk,null,AVAILABLE}
[junit4] 2> 1093782 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@5fe678e6{HTTP/1.1,[http/1.1]}{127.0.0.1:50581}
[junit4] 2> 1093782 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.e.j.s.Server Started @1112488ms
[junit4] 2> 1093782 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:59829/hdfs__localhost_59829__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-7.x_checkout_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.HdfsBasicDistributedZk2Test_197712C30A2173D9-001_tempDir-002_jetty1, replicaType=TLOG, solrconfig=solrconfig.xml, hostContext=/q_zfv/sk, hostPort=50581, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_197712C30A2173D9-001/shard-1-001/cores}
[junit4] 2> 1093782 ERROR (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1093782 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
[junit4] 2> 1093783 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1093783 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 1093783 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-09-10T23:11:13.753Z
[junit4] 2> 1093808 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1093808 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_197712C30A2173D9-001/shard-1-001/solr.xml
[junit4] 2> 1093813 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@244f7a6d, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 1093832 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53174/solr
[junit4] 2> 1093842 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [n:127.0.0.1:50581_q_zfv%2Fsk ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1093844 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [n:127.0.0.1:50581_q_zfv%2Fsk ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1093849 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [n:127.0.0.1:50581_q_zfv%2Fsk ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50581_q_zfv%2Fsk
[junit4] 2> 1093849 INFO (zkCallback-179-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1093849 INFO (zkCallback-172-thread-1-processing-n:127.0.0.1:52911_q_zfv%2Fsk) [n:127.0.0.1:52911_q_zfv%2Fsk ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1093851 INFO (zkCallback-184-thread-1-processing-n:127.0.0.1:50581_q_zfv%2Fsk) [n:127.0.0.1:50581_q_zfv%2Fsk ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1094131 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [n:127.0.0.1:50581_q_zfv%2Fsk ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@244f7a6d
[junit4] 2> 1094168 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [n:127.0.0.1:50581_q_zfv%2Fsk ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@244f7a6d
[junit4] 2> 1094168 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [n:127.0.0.1:50581_q_zfv%2Fsk ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@244f7a6d
[junit4] 2> 1094170 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[197712C30A2173D9]) [n:127.0.0.1:50581_q_zfv%2Fsk ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_197712C30A2173D9-001/shard-1-001/cores
[junit4] 2> 1094294 INFO (qtp125947171-2612) [n:127.0.0.1:50581_q_zfv%2Fsk ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:50581_q_zfv%252Fsk&action=ADDREPLICA&collection=collection1&shard=shard2&type=TLOG&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1094296 INFO (OverseerCollectionConfigSetProcessor-98637253935628292-127.0.0.1:52911_q_zfv%2Fsk-n_0000000000) [n:127.0.0.1:52911_q_zfv%2Fsk ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1094297 INFO (OverseerThreadFactory-339-thread-3-processing-n:127.0.0.1:52911_q_zfv%2Fsk) [n:127.0.0.1:52911_q_zfv%2Fsk ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:50581_q_zfv%2Fsk for creating new replica
[junit4] 2> 1094299 INFO (qtp125947171-2610) [n:127.0.0.1:50581_q_zfv%2Fsk ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_t41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=TLOG
[junit4] 2> 1094300 INFO (qtp125947171-2610) [n:127.0.0.1:50581_q_zfv%2Fsk ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 1096398 INFO (qtp125947171-2610) [n:127.0.0.1:50581_q_zfv%2Fsk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 1096560 INFO (qtp125947171-2610) [n:127.0.0.1:50581_q_zfv%2Fsk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.s.IndexSchema [collection1_shard2_replica_t41] Schema name=test
[junit4] 2> 1098600 INFO (qtp125947171-2610) [n:127.0.0.1:50581_q_zfv%2Fsk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 1098660 INFO (qtp125947171-2610) [n:127.0.0.1:50581_q_zfv%2Fsk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_t41' using configuration from collection collection1, trusted=true
[junit4] 2> 1098660 INFO (qtp125947171-2610) [n:127.0.0.1:50581_q_zfv%2Fsk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_t41' (registry 'solr.core.collection1.shard2.replica_t41') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@244f7a6d
[junit4] 2> 1098661 INFO (qtp125947171-2610) [n:127.0.0.1:50581_q_zfv%2Fsk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:59829/solr_hdfs_home
[junit4] 2> 1098662 INFO (qtp125947171-2610) [n:127.0.0.1:50581_q_zfv%2Fsk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 1098662 INFO (qtp125947171-2610) [n:127.0.0.1:50581_q_zfv%2Fsk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1098662 INFO (qtp125947171-2610) [n:127.0.0.1:50581_q_zfv%2Fsk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.SolrCore [[collection1_shard2_replica_t41] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_197712C30A2173D9-001/shard-1-001/cores/collection1_shard2_replica_t41], dataDir=[hdfs://localhost:59829/solr_hdfs_home/collection1/core_node42/data/]
[junit4] 2> 1098671 INFO (qtp125947171-2610) [n:127.0.0.1:50581_q_zfv%2Fsk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:59829/solr_hdfs_home/collection1/core_node42/data/snapshot_metadata
[junit4] 2> 1098703 INFO (qtp125947171-2610) [n:127.0.0.1:50581_q_zfv%2Fsk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1098703 INFO (qtp125947171-2610) [n:127.0.0.1:50581_q_zfv%2Fsk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 1098703 INFO (qtp125947171-2610) [n:127.0.0.1:50581_q_zfv%2Fsk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 1099712 INFO (qtp125947171-2610) [n:127.0.0.1:50581_q_zfv%2Fsk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1099719 INFO (qtp125947171-2610) [n:127.0.0.1:50581_q_zfv%2Fsk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:59829/solr_hdfs_home/collection1/core_node42/data
[junit4] 2> 1099796 INFO (qtp125947171-2610) [n:127.0.0.1:50581_q_zfv%2Fsk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:59829/solr_hdfs_home/collection1/core_node42/data/index
[junit4] 2> 1099823 INFO (qtp125947171-2610) [n:127.0.0.1:50581_q_zfv%2Fsk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1099823 INFO (qtp125947171-2610) [n:127.0.0.1:50581_q_zfv%2Fsk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 1099823 INFO (qtp125947171-2610) [n:127.0.0.1:50581_q_zfv%2Fsk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 1099891 INFO (qtp125947171-2610) [n:127.0.0.1:50581_q_zfv%2Fsk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1099892 INFO (qtp125947171-2610) [n:127.0.0.1:50581_q_zfv%2Fsk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=39, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 1100156 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:43279 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-a2bc4141-8124-40cc-a438-bb70727e7eb4:NORMAL:127.0.0.1:50454|RBW], ReplicaUC[[DISK]DS-97467601-ae00-4953-8489-c1ed5e29724c:NORMAL:127.0.0.1:43279|FINALIZED]]} size 0
[junit4] 2> 1100162 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:50454 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-a2bc4141-8124-40cc-a438-bb70727e7eb4:NORMAL:127.0.0.1:50454|RBW], ReplicaUC[[DISK]DS-97467601-ae00-4953-8489-c1ed5e29724c:NORMAL:127.0.0.1:43279|FINALIZED]]} size 0
[junit4] 2> 1100225 WARN (qtp125947171-2610) [n:127.0.0.1:50581_q_zfv%2Fsk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1100707 INFO (qtp125947171-2610) [n:127.0.0.1:50581_q_zfv%2Fsk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 1100707 INFO (qtp125947171-2610) [n:127.0.0.1:50581_q_zfv%2Fsk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1100707 INFO (qtp125947171-2610) [n:127.0.0.1:50581_q_zfv%2Fsk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 1100861 INFO (qtp125947171-2610) [n:127.0.0.1:50581_q_zfv%2Fsk c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4]
[...truncated too long message...]
> Throwable #1: java.lang.AssertionError: This doc was supposed to have been deleted, but was: SolrDocument{id=0, title_s=title0, id_i=0, inplace_updatable_float=1.0, _version_=1578205196421431296, inplace_updatable_int_with_default=666, inplace_updatable_float_with_default=42.0}
[junit4] > at __randomizedtesting.SeedInfo.seed([197712C30A2173D9:91232D19A4DD1E21]:0)
[junit4] > at org.apache.solr.update.TestInPlaceUpdatesDistrib.reorderedDBQsSimpleTest(TestInPlaceUpdatesDistrib.java:247)
[junit4] > at org.apache.solr.update.TestInPlaceUpdatesDistrib.test(TestInPlaceUpdatesDistrib.java:151)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 9801319 INFO (SUITE-TestInPlaceUpdatesDistrib-seed#[197712C30A2173D9]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> 9801319 INFO (SUITE-TestInPlaceUpdatesDistrib-seed#[197712C30A2173D9]-worker) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=169231976
[junit4] 2> 9801319 INFO (SUITE-TestInPlaceUpdatesDistrib-seed#[197712C30A2173D9]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 9801319 INFO (SUITE-TestInPlaceUpdatesDistrib-seed#[197712C30A2173D9]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6180cfb4: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@73d6be83
[junit4] 2> 9801320 INFO (SUITE-TestInPlaceUpdatesDistrib-seed#[197712C30A2173D9]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 9801320 INFO (SUITE-TestInPlaceUpdatesDistrib-seed#[197712C30A2173D9]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2bf506aa: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@27b8c7de
[junit4] 2> 9801320 INFO (SUITE-TestInPlaceUpdatesDistrib-seed#[197712C30A2173D9]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 9801320 INFO (SUITE-TestInPlaceUpdatesDistrib-seed#[197712C30A2173D9]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@c15fd59: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@65b81d94
[junit4] 2> 9801321 INFO (SUITE-TestInPlaceUpdatesDistrib-seed#[197712C30A2173D9]-worker) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 9801321 INFO (coreCloseExecutor-10486-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@1ae64174
[junit4] 2> 9801321 INFO (coreCloseExecutor-10486-thread-1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=451297652
[junit4] 2> 9801321 INFO (coreCloseExecutor-10486-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@17bf71d1: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@40af0062
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.update.TestInPlaceUpdatesDistrib_197712C30A2173D9-001
[junit4] 2> Sep 11, 2017 1:36:11 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 3 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarity(queryNorm=false): {}, locale=tr-TR, timezone=Asia/Kolkata
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=219845656,total=522715136
[junit4] 2> NOTE: All tests run in this JVM: [TestHttpShardHandlerFactory, TestExceedMaxTermLength, TestStressInPlaceUpdates, TestManagedSchema, DistributedQueryComponentCustomSortTest, TestZkAclsWithHadoopAuth, ZkSolrClientTest, SoftAutoCommitTest, TestSweetSpotSimilarityFactory, ShardRoutingCustomTest, SuggestComponentTest, SimpleCollectionCreateDeleteTest, LeaderInitiatedRecoveryOnShardRestartTest, TestSolrXml, TestConfigSets, TestOverriddenPrefixQueryForCustomFieldType, TestXmlQParser, TestConfigReload, TestSortByMinMaxFunction, ChaosMonkeyNothingIsSafeWithPullReplicasTest, TestShardHandlerFactory, TestBulkSchemaAPI, ParsingFieldUpdateProcessorsTest, MultiThreadedOCPTest, LeaderFailoverAfterPartitionTest, HdfsBasicDistributedZkTest, QueryParsingTest, DistributedIntervalFacetingTest, SpatialRPTFieldTypeTest, TestDynamicLoading, TestPartialUpdateDeduplication, TestRequestStatusCollectionAPI, TestLeaderElectionWithEmptyReplica, TestCoreContainer, TestManagedResource, AnalysisAfterCoreReloadTest, TestReqParamsAPI, IndexSchemaTest, TestSlowCompositeReaderWrapper, TestDistributedGrouping, ShufflingReplicaListTransformerTest, DocValuesMultiTest, TestSubQueryTransformer, TestReRankQParserPlugin, SystemInfoHandlerTest, BlockJoinFacetRandomTest, TestRebalanceLeaders, TestStandardQParsers, TestSolrJ, TestSearcherReuse, CacheHeaderTest, TestStressRecovery, DateRangeFieldTest, TestRawTransformer, TestSystemCollAutoCreate, DeleteInactiveReplicaTest, InfoHandlerTest, TestHashPartitioner, TestQueryWrapperFilter, TestCloudSchemaless, DisMaxRequestHandlerTest, CollectionsAPIDistributedZkTest, TestManagedSynonymFilterFactory, TestSolrIndexConfig, CSVRequestHandlerTest, TestNumericTerms64, TestRTGBase, TestLRUCache, NoCacheHeaderTest, ZkStateReaderTest, DistributedFacetPivotSmallAdvancedTest, CloneFieldUpdateProcessorFactoryTest, TermVectorComponentTest, TestBM25SimilarityFactory, ResourceLoaderTest, CollectionsAPISolrJTest, TestBlendedInfixSuggestions, TestAddFieldRealTimeGet, DistributedMLTComponentTest, DistribCursorPagingTest, ZkCLITest, RequestLoggingTest, TestDefaultStatsCache, OverriddenZkACLAndCredentialsProvidersTest, TemplateUpdateProcessorTest, ForceLeaderTest, DirectSolrSpellCheckerTest, SampleTest, TestPostingsSolrHighlighter, TestValueSourceCache, TestFilteredDocIdSet, TestExportWriter, TestSerializedLuceneMatchVersion, TestInPlaceUpdatesStandalone, TestCursorMarkWithoutUniqueKey, HdfsRecoveryZkTest, FileBasedSpellCheckerTest, SolrXmlInZkTest, TestFieldCacheWithThreads, SOLR749Test, SolrCmdDistributorTest, IndexSchemaRuntimeFieldTest, TestObjectReleaseTracker, OpenCloseCoreStressTest, CheckHdfsIndexTest, HdfsDirectoryFactoryTest, SuggesterTSTTest, FileUtilsTest, ClusterStateUpdateTest, HdfsThreadLeakTest, BigEndianAscendingWordDeserializerTest, TestCloudPseudoReturnFields, TestNonDefinedSimilarityFactory, LoggingHandlerTest, MoveReplicaHDFSTest, TestCoreDiscovery, TestSolr4Spatial2, SimpleMLTQParserTest, StatelessScriptUpdateProcessorFactoryTest, TestPKIAuthenticationPlugin, ShardSplitTest, TestStressVersions, SimpleFacetsTest, SpellCheckComponentTest, BadIndexSchemaTest, ConvertedLegacyTest, TestSort, TestIndexSearcher, TestExtendedDismaxParser, SolrRequestParserTest, TestFoldingMultitermQuery, DocValuesTest, TestQueryTypes, DocumentBuilderTest, MoreLikeThisHandlerTest, UpdateRequestProcessorFactoryTest, TestAnalyzedSuggestions, BinaryUpdateRequestHandlerTest, TestComponentsName, HighlighterConfigTest, TestQuerySenderListener, AlternateDirectoryTest, BadComponentTest, TestMergePolicyConfig, TestSolrDeletionPolicy2, TestSearchPerf, TestElisionMultitermQuery, OutputWriterTest, TestSolrCoreProperties, TestLuceneMatchVersion, SynonymTokenizerTest, TestPerFieldSimilarity, TimeZoneUtilsTest, TestFastOutputStream, TestFastLRUCache, PrimUtilsTest, DateFieldTest, TestEmbeddedSolrServerAdminHandler, TestEmbeddedSolrServerSchemaAPI, ConnectionReuseTest, AssignTest, CdcrVersionReplicationTest, CloudExitableDirectoryReaderTest, CollectionReloadTest, CollectionTooManyReplicasTest, ConfigSetsAPITest, ConnectionManagerTest, DeleteNodeTest, DeleteReplicaTest, DistributedVersionInfoTest, OverseerModifyCollectionTest, OverseerStatusTest, ReplaceNodeTest, SSLMigrationTest, SharedFSAutoReplicaFailoverTest, TestCloudSearcherWarming, TestClusterProperties, TestCollectionsAPIViaSolrCloudCluster, TestConfigSetsAPI, TestLeaderInitiatedRecoveryThread, TestLockTree, TestMiniSolrCloudClusterSSL, TestPullReplica, TestReplicaProperties, TestSSLRandomization, TestShortCircuitedRequests, TestSolrCloudWithDelegationTokens, TestSolrCloudWithKerberosAlt, TestTolerantUpdateProcessorCloud, TestTolerantUpdateProcessorRandomCloud, TlogReplayBufferedWhileIndexingTest, HdfsSyncSliceTest, StressHdfsTest, TestSolrConfigHandlerConcurrent, V2ApiIntegrationTest, CoreAdminCreateDiscoverTest, CoreAdminRequestStatusTest, CoreMergeIndexesAdminHandlerTest, SecurityConfHandlerTest, StatsReloadRaceTest, DistributedFacetExistsSmallTest, DistributedSuggestComponentTest, TestUnifiedSolrHighlighter, TestMultiValuedNumericRangeQuery, TestNumericRangeQuery64, SubstringBytesRefFilterTest, TestIntervalFaceting, TestLegacyFieldCache, TestNumericTerms32, TestUninvertingReader, AnalysisErrorHandlingTest, CdcrUpdateLogTest, PeerSyncWithIndexFingerprintCachingTest, TestDocBasedVersionConstraints, TestInPlaceUpdatesDistrib]
[junit4] Completed [731/731 (8!)] on J1 in 208.95s, 1 test, 1 failure <<< FAILURES!
[...truncated 46200 lines...]
[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 42 - Failure
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/42/
8 tests failed.
FAILED: org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader
Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:40196/_/m/forceleader_test_collection
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:40196/_/m/forceleader_test_collection
at __randomizedtesting.SeedInfo.seed([101D748BEEB1175D:F68A404BD733EE3C]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:638)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at org.apache.solr.cloud.HttpPartitionTest.realTimeGetDocId(HttpPartitionTest.java:618)
at org.apache.solr.cloud.HttpPartitionTest.assertDocExists(HttpPartitionTest.java:603)
at org.apache.solr.cloud.HttpPartitionTest.assertDocsExistInAllReplicas(HttpPartitionTest.java:558)
at org.apache.solr.cloud.ForceLeaderTest.bringBackOldLeaderAndSendDoc(ForceLeaderTest.java:392)
at org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader(ForceLeaderTest.java:151)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:526)
... 49 more
FAILED: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test
Error Message:
Java heap space
Stack Trace:
java.lang.OutOfMemoryError: Java heap space
FAILED: org.apache.solr.cloud.RollingRestartTest.test
Error Message:
KeeperErrorCode = Session expired for /overseer_elect/leader
Stack Trace:
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
at __randomizedtesting.SeedInfo.seed([101D748BEEB1175D:98494B51404D7AA5]:0)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1212)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:357)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:354)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:354)
at org.apache.solr.cloud.OverseerTaskProcessor.getLeaderId(OverseerTaskProcessor.java:374)
at org.apache.solr.cloud.OverseerTaskProcessor.getLeaderNode(OverseerTaskProcessor.java:365)
at org.apache.solr.cloud.RollingRestartTest.waitUntilOverseerDesignateIsLeader(RollingRestartTest.java:137)
at org.apache.solr.cloud.RollingRestartTest.restartWithRolesTest(RollingRestartTest.java:104)
at org.apache.solr.cloud.RollingRestartTest.test(RollingRestartTest.java:52)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_dv
Error Message:
java.lang.RuntimeException: Error from server at http://127.0.0.1:44116/solr/test_col: Async exception during distributed update: Error from server at http://127.0.0.1:45284/solr/test_col_shard5_replica_n18: Server Error request: http://127.0.0.1:45284/solr/test_col_shard5_replica_n18/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A44116%2Fsolr%2Ftest_col_shard3_replica_n10%2F&wt=javabin&version=2 Remote error message: Failed synchronous update on shard StdNode: http://127.0.0.1:35862/solr/test_col_shard5_replica_n17/ update: org.apache.solr.client.solrj.request.UpdateRequest@5ad5ff69
Stack Trace:
java.util.concurrent.ExecutionException: java.lang.RuntimeException: Error from server at http://127.0.0.1:44116/solr/test_col: Async exception during distributed update: Error from server at http://127.0.0.1:45284/solr/test_col_shard5_replica_n18: Server Error
request: http://127.0.0.1:45284/solr/test_col_shard5_replica_n18/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A44116%2Fsolr%2Ftest_col_shard3_replica_n10%2F&wt=javabin&version=2
Remote error message: Failed synchronous update on shard StdNode: http://127.0.0.1:35862/solr/test_col_shard5_replica_n17/ update: org.apache.solr.client.solrj.request.UpdateRequest@5ad5ff69
at __randomizedtesting.SeedInfo.seed([101D748BEEB1175D:260916CD64EC2D4C]:0)
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.checkField(TestStressCloudBlindAtomicUpdates.java:283)
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_dv(TestStressCloudBlindAtomicUpdates.java:195)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: Error from server at http://127.0.0.1:44116/solr/test_col: Async exception during distributed update: Error from server at http://127.0.0.1:45284/solr/test_col_shard5_replica_n18: Server Error
request: http://127.0.0.1:45284/solr/test_col_shard5_replica_n18/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A44116%2Fsolr%2Ftest_col_shard3_replica_n10%2F&wt=javabin&version=2
Remote error message: Failed synchronous update on shard StdNode: http://127.0.0.1:35862/solr/test_col_shard5_replica_n17/ update: org.apache.solr.client.solrj.request.UpdateRequest@5ad5ff69
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.run(TestStressCloudBlindAtomicUpdates.java:411)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:44116/solr/test_col: Async exception during distributed update: Error from server at http://127.0.0.1:45284/solr/test_col_shard5_replica_n18: Server Error
request: http://127.0.0.1:45284/solr/test_col_shard5_replica_n18/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A44116%2Fsolr%2Ftest_col_shard3_replica_n10%2F&wt=javabin&version=2
Remote error message: Failed synchronous update on shard StdNode: http://127.0.0.1:35862/solr/test_col_shard5_replica_n17/ update: org.apache.solr.client.solrj.request.UpdateRequest@5ad5ff69
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:627)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195)
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.doRandomAtomicUpdate(TestStressCloudBlindAtomicUpdates.java:370)
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.run(TestStressCloudBlindAtomicUpdates.java:399)
... 6 more
FAILED: org.apache.solr.cloud.hdfs.HdfsBasicDistributedZk2Test.test
Error Message:
.response.numFound:36!=67
Stack Trace:
junit.framework.AssertionFailedError: .response.numFound:36!=67
at __randomizedtesting.SeedInfo.seed([101D748BEEB1175D:98494B51404D7AA5]:0)
at junit.framework.Assert.fail(Assert.java:50)
at org.apache.solr.BaseDistributedSearchTestCase.compareSolrResponses(BaseDistributedSearchTestCase.java:913)
at org.apache.solr.BaseDistributedSearchTestCase.compareResponses(BaseDistributedSearchTestCase.java:940)
at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:607)
at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:587)
at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:566)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkQueries(AbstractFullDistribZkTestBase.java:937)
at org.apache.solr.cloud.BasicDistributedZk2Test.test(BasicDistributedZk2Test.java:102)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.hdfs.HdfsRestartWhileUpdatingTest.test
Error Message:
shard1 is not consistent. Got 1641 from http://127.0.0.1:35477/collection1_shard1_replica_n21 (previous client) and got 1567 from http://127.0.0.1:36236/collection1_shard1_replica_n23
Stack Trace:
java.lang.AssertionError: shard1 is not consistent. Got 1641 from http://127.0.0.1:35477/collection1_shard1_replica_n21 (previous client) and got 1567 from http://127.0.0.1:36236/collection1_shard1_replica_n23
at __randomizedtesting.SeedInfo.seed([101D748BEEB1175D:98494B51404D7AA5]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1323)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1302)
at org.apache.solr.cloud.RestartWhileUpdatingTest.test(RestartWhileUpdatingTest.java:155)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsRestartWhileUpdatingTest
Error Message:
5 threads leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsRestartWhileUpdatingTest: 1) Thread[id=3537, name=searcherExecutor-1237-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=4368, name=searcherExecutor-1313-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=3571, name=searcherExecutor-1248-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=4339, name=searcherExecutor-1302-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=4311, name=searcherExecutor-1291-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 5 threads leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsRestartWhileUpdatingTest:
1) Thread[id=3537, name=searcherExecutor-1237-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=4368, name=searcherExecutor-1313-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=3571, name=searcherExecutor-1248-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
4) Thread[id=4339, name=searcherExecutor-1302-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
5) Thread[id=4311, name=searcherExecutor-1291-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([101D748BEEB1175D]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsRestartWhileUpdatingTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=3537, name=searcherExecutor-1237-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=4368, name=searcherExecutor-1313-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=3571, name=searcherExecutor-1248-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=4339, name=searcherExecutor-1302-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=4311, name=searcherExecutor-1291-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=3537, name=searcherExecutor-1237-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=4368, name=searcherExecutor-1313-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=3571, name=searcherExecutor-1248-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
4) Thread[id=4339, name=searcherExecutor-1302-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
5) Thread[id=4311, name=searcherExecutor-1291-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([101D748BEEB1175D]:0)
Build Log:
[...truncated 12263 lines...]
[junit4] Suite: org.apache.solr.cloud.hdfs.HdfsBasicDistributedZk2Test
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_101D748BEEB1175D-001/init-core-data-001
[junit4] 2> 96138 WARN (SUITE-HdfsBasicDistributedZk2Test-seed#[101D748BEEB1175D]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=34 numCloses=34
[junit4] 2> 96138 INFO (SUITE-HdfsBasicDistributedZk2Test-seed#[101D748BEEB1175D]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 96140 INFO (SUITE-HdfsBasicDistributedZk2Test-seed#[101D748BEEB1175D]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
[junit4] 2> 96140 INFO (SUITE-HdfsBasicDistributedZk2Test-seed#[101D748BEEB1175D]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 96140 INFO (SUITE-HdfsBasicDistributedZk2Test-seed#[101D748BEEB1175D]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /bscys/
[junit4] 2> 100216 WARN (SUITE-HdfsBasicDistributedZk2Test-seed#[101D748BEEB1175D]-worker) [ ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 103122 WARN (SUITE-HdfsBasicDistributedZk2Test-seed#[101D748BEEB1175D]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 104542 INFO (SUITE-HdfsBasicDistributedZk2Test-seed#[101D748BEEB1175D]-worker) [ ] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
[junit4] 2> 104803 WARN (SUITE-HdfsBasicDistributedZk2Test-seed#[101D748BEEB1175D]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 107739 INFO (SUITE-HdfsBasicDistributedZk2Test-seed#[101D748BEEB1175D]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 108368 INFO (SUITE-HdfsBasicDistributedZk2Test-seed#[101D748BEEB1175D]-worker) [ ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/hdfs to ./temp/Jetty_lucene2.us.west_apache_org_44196_hdfs____.1701e0/webapp
[junit4] 2> 111758 INFO (SUITE-HdfsBasicDistributedZk2Test-seed#[101D748BEEB1175D]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@lucene2-us-west.apache.org:44196
[junit4] 2> 120570 WARN (SUITE-HdfsBasicDistributedZk2Test-seed#[101D748BEEB1175D]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 120605 INFO (SUITE-HdfsBasicDistributedZk2Test-seed#[101D748BEEB1175D]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 120684 INFO (SUITE-HdfsBasicDistributedZk2Test-seed#[101D748BEEB1175D]-worker) [ ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_43636_datanode____c6a90h/webapp
[junit4] 2> 122485 INFO (SUITE-HdfsBasicDistributedZk2Test-seed#[101D748BEEB1175D]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:43636
[junit4] 2> 124819 WARN (SUITE-HdfsBasicDistributedZk2Test-seed#[101D748BEEB1175D]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 124853 INFO (SUITE-HdfsBasicDistributedZk2Test-seed#[101D748BEEB1175D]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 124997 INFO (SUITE-HdfsBasicDistributedZk2Test-seed#[101D748BEEB1175D]-worker) [ ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_46214_datanode____.ffchae/webapp
[junit4] 2> 127184 INFO (SUITE-HdfsBasicDistributedZk2Test-seed#[101D748BEEB1175D]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:46214
[junit4] 2> 131069 ERROR (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_101D748BEEB1175D-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_101D748BEEB1175D-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:38550) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 131101 ERROR (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_101D748BEEB1175D-001/tempDir-001/hdfsBaseDir/data/data3/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_101D748BEEB1175D-001/tempDir-001/hdfsBaseDir/data/data4/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:38550) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 132232 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x122c07752f6e4b: from storage DS-fc35deec-b3b8-450f-ad1b-8e7ec26ab05f node DatanodeRegistration(127.0.0.1:42070, datanodeUuid=06f325b4-d3c8-4175-aca8-d8824979e363, infoPort=40342, infoSecurePort=0, ipcPort=37696, storageInfo=lv=-56;cid=testClusterID;nsid=706439863;c=0), blocks: 0, hasStaleStorage: true, processing time: 16 msecs
[junit4] 2> 132233 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x122c0774462445: from storage DS-03403586-5ca0-4d81-8eaa-042ed7ac9590 node DatanodeRegistration(127.0.0.1:45129, datanodeUuid=da630535-83bc-4150-8666-bd6ad1d7cd73, infoPort=42172, infoSecurePort=0, ipcPort=37207, storageInfo=lv=-56;cid=testClusterID;nsid=706439863;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 132233 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x122c07752f6e4b: from storage DS-6a145b12-32d3-49cf-bfd7-d8b8f982b4ca node DatanodeRegistration(127.0.0.1:42070, datanodeUuid=06f325b4-d3c8-4175-aca8-d8824979e363, infoPort=40342, infoSecurePort=0, ipcPort=37696, storageInfo=lv=-56;cid=testClusterID;nsid=706439863;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 132248 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x122c0774462445: from storage DS-e56d0b8e-7760-497b-8027-351898e1e1e6 node DatanodeRegistration(127.0.0.1:45129, datanodeUuid=da630535-83bc-4150-8666-bd6ad1d7cd73, infoPort=42172, infoSecurePort=0, ipcPort=37207, storageInfo=lv=-56;cid=testClusterID;nsid=706439863;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 133891 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 133912 INFO (Thread-186) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 133912 INFO (Thread-186) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 133923 ERROR (Thread-186) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 134020 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.a.s.c.ZkTestServer start zk server on port:39105
[junit4] 2> 134261 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 134263 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 134297 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 134298 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 134299 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 134342 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 134344 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 134360 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 134406 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 134408 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 134408 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 134512 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use TLOG replicas unless explicitly asked otherwise
[junit4] 2> 136735 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 136820 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@49ade18b{/bscys,null,AVAILABLE}
[junit4] 2> 136821 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3b40b0e7{HTTP/1.1,[http/1.1]}{127.0.0.1:32969}
[junit4] 2> 136821 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.e.j.s.Server Started @147891ms
[junit4] 2> 136821 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://lucene2-us-west.apache.org:38550/hdfs__lucene2-us-west.apache.org_38550__home_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-7.x_checkout_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.HdfsBasicDistributedZk2Test_101D748BEEB1175D-001_tempDir-002_control_data, replicaType=NRT, hostContext=/bscys, hostPort=32969, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_101D748BEEB1175D-001/control-001/cores}
[junit4] 2> 136821 ERROR (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 136821 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
[junit4] 2> 136822 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 136822 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 136822 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-09-10T01:07:35.499Z
[junit4] 2> 136966 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 136966 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_101D748BEEB1175D-001/control-001/solr.xml
[junit4] 2> 137064 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@445ef360, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 137194 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39105/solr
[junit4] 2> 138139 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [n:127.0.0.1:32969_bscys ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 138140 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [n:127.0.0.1:32969_bscys ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:32969_bscys
[junit4] 2> 138153 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [n:127.0.0.1:32969_bscys ] o.a.s.c.Overseer Overseer (id=98632050448793604-127.0.0.1:32969_bscys-n_0000000000) starting
[junit4] 2> 138213 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [n:127.0.0.1:32969_bscys ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:32969_bscys
[junit4] 2> 138274 INFO (OverseerStateUpdate-98632050448793604-127.0.0.1:32969_bscys-n_0000000000) [n:127.0.0.1:32969_bscys ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 139421 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [n:127.0.0.1:32969_bscys ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@445ef360
[junit4] 2> 139489 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [n:127.0.0.1:32969_bscys ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@445ef360
[junit4] 2> 139517 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [n:127.0.0.1:32969_bscys ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@445ef360
[junit4] 2> 139534 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [n:127.0.0.1:32969_bscys ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_101D748BEEB1175D-001/control-001/cores
[junit4] 2> 139929 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 139930 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:39105/solr ready
[junit4] 2> 139948 INFO (qtp1897712179-526) [n:127.0.0.1:32969_bscys ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:32969_bscys&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 140042 INFO (OverseerThreadFactory-101-thread-1-processing-n:127.0.0.1:32969_bscys) [n:127.0.0.1:32969_bscys ] o.a.s.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 140194 INFO (qtp1897712179-523) [n:127.0.0.1:32969_bscys ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 140194 INFO (qtp1897712179-523) [n:127.0.0.1:32969_bscys ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 140537 INFO (zkCallback-63-thread-1-processing-n:127.0.0.1:32969_bscys) [n:127.0.0.1:32969_bscys ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 141533 INFO (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 142221 INFO (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
[junit4] 2> 145296 WARN (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.WordDelimiterFilterFactory]. Please consult documentation how to replace it accordingly.
[junit4] 2> 145666 WARN (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.GeoHashField]. Please consult documentation how to replace it accordingly.
[junit4] 2> 145713 WARN (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.LatLonType]. Please consult documentation how to replace it accordingly.
[junit4] 2> 145731 WARN (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.CurrencyField]. Please consult documentation how to replace it accordingly.
[junit4] 2> 146307 INFO (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 146836 INFO (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
[junit4] 2> 146837 INFO (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@445ef360
[junit4] 2> 146916 INFO (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://lucene2-us-west.apache.org:38550/solr_hdfs_home
[junit4] 2> 146916 INFO (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 146916 INFO (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 146916 INFO (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_101D748BEEB1175D-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[hdfs://lucene2-us-west.apache.org:38550/solr_hdfs_home/control_collection/core_node2/data/]
[junit4] 2> 146918 INFO (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:38550/solr_hdfs_home/control_collection/core_node2/data/snapshot_metadata
[junit4] 2> 147064 INFO (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 147064 INFO (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 147064 INFO (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Creating new global HDFS BlockCache
[junit4] 2> 151832 INFO (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 151862 INFO (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:38550/solr_hdfs_home/control_collection/core_node2/data
[junit4] 2> 152370 INFO (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:38550/solr_hdfs_home/control_collection/core_node2/data/index
[junit4] 2> 152469 INFO (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 152469 INFO (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 152545 INFO (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 152546 INFO (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=33, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=1.333984375, noCFSRatio=1.0]
[junit4] 2> 154837 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:45129 is added to blk_1073741825_1001{UCState=COMMITTED, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-fc35deec-b3b8-450f-ad1b-8e7ec26ab05f:NORMAL:127.0.0.1:42070|RBW], ReplicaUC[[DISK]DS-e56d0b8e-7760-497b-8027-351898e1e1e6:NORMAL:127.0.0.1:45129|RBW]]} size 72
[junit4] 2> 154837 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:42070 is added to blk_1073741825_1001 size 72
[junit4] 2> 155667 WARN (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 156258 INFO (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 156259 INFO (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 156259 INFO (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 156357 INFO (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 156358 INFO (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 156394 INFO (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@5df8cd27
[junit4] 2> 157395 INFO (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@18d8e1a2[control_collection_shard1_replica_n1] main]
[junit4] 2> 157634 INFO (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 157635 INFO (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 157702 INFO (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 157869 INFO (searcherExecutor-104-thread-1-processing-n:127.0.0.1:32969_bscys x:control_collection_shard1_replica_n1 s:shard1 c:control_collection) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@18d8e1a2[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 157869 INFO (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1578112832289898496
[junit4] 2> 157934 INFO (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 157934 INFO (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 157934 INFO (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:32969/bscys/control_collection_shard1_replica_n1/
[junit4] 2> 157934 INFO (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 157934 INFO (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:32969/bscys/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 157934 INFO (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 157981 INFO (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:32969/bscys/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 157996 INFO (zkCallback-63-thread-1-processing-n:127.0.0.1:32969_bscys) [n:127.0.0.1:32969_bscys ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 158110 INFO (zkCallback-63-thread-1-processing-n:127.0.0.1:32969_bscys) [n:127.0.0.1:32969_bscys ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 158151 INFO (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 158155 INFO (qtp1897712179-523) [n:127.0.0.1:32969_bscys c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=17961
[junit4] 2> 158232 INFO (qtp1897712179-526) [n:127.0.0.1:32969_bscys ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 158296 INFO (zkCallback-63-thread-1-processing-n:127.0.0.1:32969_bscys) [n:127.0.0.1:32969_bscys ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 159232 INFO (qtp1897712179-526) [n:127.0.0.1:32969_bscys ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:32969_bscys&wt=javabin&version=2} status=0 QTime=19283
[junit4] 2> 159759 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 159760 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:39105/solr ready
[junit4] 2> 159762 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 159824 INFO (qtp1897712179-527) [n:127.0.0.1:32969_bscys ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 159954 INFO (OverseerThreadFactory-101-thread-2-processing-n:127.0.0.1:32969_bscys) [n:127.0.0.1:32969_bscys ] o.a.s.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 159954 INFO (OverseerCollectionConfigSetProcessor-98632050448793604-127.0.0.1:32969_bscys-n_0000000000) [n:127.0.0.1:32969_bscys ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 159955 WARN (OverseerThreadFactory-101-thread-2-processing-n:127.0.0.1:32969_bscys) [n:127.0.0.1:32969_bscys ] o.a.s.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
[junit4] 2> 160184 INFO (qtp1897712179-527) [n:127.0.0.1:32969_bscys ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 160184 INFO (qtp1897712179-527) [n:127.0.0.1:32969_bscys ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=1&wt=javabin&version=2} status=0 QTime=360
[junit4] 2> 161690 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_101D748BEEB1175D-001/shard-1-001 of type TLOG
[junit4] 2> 161709 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 161818 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5142b610{/bscys,null,AVAILABLE}
[junit4] 2> 161819 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@48bccbba{HTTP/1.1,[http/1.1]}{127.0.0.1:39218}
[junit4] 2> 161819 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.e.j.s.Server Started @172889ms
[junit4] 2> 161819 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://lucene2-us-west.apache.org:38550/hdfs__lucene2-us-west.apache.org_38550__home_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-7.x_checkout_solr_build_solr-core_test_J0_temp_solr.cloud.hdfs.HdfsBasicDistributedZk2Test_101D748BEEB1175D-001_tempDir-002_jetty1, replicaType=TLOG, solrconfig=solrconfig.xml, hostContext=/bscys, hostPort=39218, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_101D748BEEB1175D-001/shard-1-001/cores}
[junit4] 2> 161819 ERROR (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 161820 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
[junit4] 2> 161820 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 161820 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 161820 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-09-10T01:08:00.497Z
[junit4] 2> 161985 INFO (OverseerCollectionConfigSetProcessor-98632050448793604-127.0.0.1:32969_bscys-n_0000000000) [n:127.0.0.1:32969_bscys ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 162037 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 162037 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZk2Test_101D748BEEB1175D-001/shard-1-001/solr.xml
[junit4] 2> 162133 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@445ef360, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 162229 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39105/solr
[junit4] 2> 162560 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [n:127.0.0.1:39218_bscys ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 162575 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [n:127.0.0.1:39218_bscys ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 162576 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [n:127.0.0.1:39218_bscys ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39218_bscys
[junit4] 2> 162772 INFO (zkCallback-75-thread-1-processing-n:127.0.0.1:39218_bscys) [n:127.0.0.1:39218_bscys ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 162785 INFO (zkCallback-70-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 162801 INFO (zkCallback-63-thread-1-processing-n:127.0.0.1:32969_bscys) [n:127.0.0.1:32969_bscys ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 163689 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [n:127.0.0.1:39218_bscys ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@445ef360
[junit4] 2> 163874 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [n:127.0.0.1:39218_bscys ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@445ef360
[junit4] 2> 163874 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [n:127.0.0.1:39218_bscys ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@445ef360
[junit4] 2> 163876 INFO (TEST-HdfsBasicDistributedZk2Test.test-seed#[101D748BEEB1175D]) [n:127.0.0.1:39218_bscys ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x
[...truncated too long message...]
eadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 2) Thread[id=4368, name=searcherExecutor-1313-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 3) Thread[id=3571, name=searcherExecutor-1248-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 4) Thread[id=4339, name=searcherExecutor-1302-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 5) Thread[id=4311, name=searcherExecutor-1291-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([101D748BEEB1175D]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=3537, name=searcherExecutor-1237-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 2) Thread[id=4368, name=searcherExecutor-1313-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 3) Thread[id=3571, name=searcherExecutor-1248-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 4) Thread[id=4339, name=searcherExecutor-1302-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 5) Thread[id=4311, name=searcherExecutor-1291-thread-1, state=WAITING, group=TGRP-HdfsRestartWhileUpdatingTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([101D748BEEB1175D]:0)
[junit4] Completed [292/731 (6!)] on J2 in 598.63s, 1 test, 1 failure, 2 errors <<< FAILURES!
[...truncated 1334 lines...]
[junit4] JVM J2: stdout was not empty, see: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/temp/junit4-J2-20170910_010507_411886280216993649752.sysout
[junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
[junit4] java.lang.OutOfMemoryError: Java heap space
[junit4] Dumping heap to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/heapdumps/java_pid1425.hprof ...
[junit4] Heap dump file created [587368194 bytes in 18.826 secs]
[junit4] <<< JVM J2: EOF ----
[...truncated 8203 lines...]
BUILD FAILED
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/build.xml:834: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/build.xml:778: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid1425.hprof
Total time: 668 minutes 39 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 41 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/41/
4 tests failed.
FAILED: org.apache.solr.cloud.HttpPartitionTest.test
Error Message:
Doc with id=894 not found in http://127.0.0.1:45166/c8n_1x2 due to: Path not found: /id; rsp={doc=null}
Stack Trace:
java.lang.AssertionError: Doc with id=894 not found in http://127.0.0.1:45166/c8n_1x2 due to: Path not found: /id; rsp={doc=null}
at __randomizedtesting.SeedInfo.seed([EDB3304C08F978F6:65E70F96A605150E]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.HttpPartitionTest.assertDocExists(HttpPartitionTest.java:605)
at org.apache.solr.cloud.HttpPartitionTest.assertDocsExistInAllReplicas(HttpPartitionTest.java:560)
at org.apache.solr.cloud.HttpPartitionTest.testRf2(HttpPartitionTest.java:381)
at org.apache.solr.cloud.HttpPartitionTest.test(HttpPartitionTest.java:132)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.LeaderElectionContextKeyTest.test
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([EDB3304C08F978F6:65E70F96A605150E]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.cloud.LeaderElectionContextKeyTest.test(LeaderElectionContextKeyTest.java:88)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.core.HdfsDirectoryFactoryTest.testInitArgsOrSysPropConfig
Error Message:
The max direct memory is likely too low. Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~134.217728 MB.
Stack Trace:
java.lang.RuntimeException: The max direct memory is likely too low. Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~134.217728 MB.
at __randomizedtesting.SeedInfo.seed([EDB3304C08F978F6:1A1CF967F57092DD]:0)
at org.apache.solr.core.HdfsDirectoryFactory.createBlockCache(HdfsDirectoryFactory.java:311)
at org.apache.solr.core.HdfsDirectoryFactory.getBlockDirectoryCache(HdfsDirectoryFactory.java:287)
at org.apache.solr.core.HdfsDirectoryFactory.create(HdfsDirectoryFactory.java:227)
at org.apache.solr.core.HdfsDirectoryFactoryTest.testInitArgsOrSysPropConfig(HdfsDirectoryFactoryTest.java:111)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Bits.java:694)
at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
at org.apache.solr.store.blockcache.BlockCache.<init>(BlockCache.java:71)
at org.apache.solr.core.HdfsDirectoryFactory.createBlockCache(HdfsDirectoryFactory.java:309)
... 42 more
FAILED: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test
Error Message:
Timeout waiting for all live and active
Stack Trace:
java.lang.AssertionError: Timeout waiting for all live and active
at __randomizedtesting.SeedInfo.seed([EDB3304C08F978F6:65E70F96A605150E]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.testBasics(SharedFSAutoReplicaFailoverTest.java:317)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test(SharedFSAutoReplicaFailoverTest.java:136)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 13649 lines...]
[junit4] Suite: org.apache.solr.cloud.LeaderElectionContextKeyTest
[junit4] 2> 4481162 INFO (SUITE-LeaderElectionContextKeyTest-seed#[EDB3304C08F978F6]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderElectionContextKeyTest_EDB3304C08F978F6-001/init-core-data-001
[junit4] 2> 4481163 WARN (SUITE-LeaderElectionContextKeyTest-seed#[EDB3304C08F978F6]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=65 numCloses=65
[junit4] 2> 4481163 INFO (SUITE-LeaderElectionContextKeyTest-seed#[EDB3304C08F978F6]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 4481172 INFO (SUITE-LeaderElectionContextKeyTest-seed#[EDB3304C08F978F6]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 4481172 INFO (SUITE-LeaderElectionContextKeyTest-seed#[EDB3304C08F978F6]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderElectionContextKeyTest_EDB3304C08F978F6-001/tempDir-001
[junit4] 2> 4481172 INFO (SUITE-LeaderElectionContextKeyTest-seed#[EDB3304C08F978F6]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 4481173 INFO (Thread-73365) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 4481173 INFO (Thread-73365) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 4481177 ERROR (Thread-73365) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 4481273 INFO (SUITE-LeaderElectionContextKeyTest-seed#[EDB3304C08F978F6]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:45185
[junit4] 2> 4481291 INFO (jetty-launcher-3991-thread-1) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 4481292 INFO (jetty-launcher-3991-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@c1675b2{/solr,null,AVAILABLE}
[junit4] 2> 4481292 INFO (jetty-launcher-3991-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@5c4f747d{HTTP/1.1,[http/1.1]}{127.0.0.1:55455}
[junit4] 2> 4481292 INFO (jetty-launcher-3991-thread-1) [ ] o.e.j.s.Server Started @4489760ms
[junit4] 2> 4481292 INFO (jetty-launcher-3991-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=55455}
[junit4] 2> 4481293 ERROR (jetty-launcher-3991-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 4481293 INFO (jetty-launcher-3991-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.1.0
[junit4] 2> 4481293 INFO (jetty-launcher-3991-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 4481293 INFO (jetty-launcher-3991-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 4481293 INFO (jetty-launcher-3991-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-09-08T20:09:31.497Z
[junit4] 2> 4481301 INFO (jetty-launcher-3991-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 4481308 INFO (jetty-launcher-3991-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45185/solr
[junit4] 2> 4481348 INFO (jetty-launcher-3991-thread-1) [n:127.0.0.1:55455_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 4481348 INFO (jetty-launcher-3991-thread-1) [n:127.0.0.1:55455_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:55455_solr
[junit4] 2> 4481348 INFO (jetty-launcher-3991-thread-1) [n:127.0.0.1:55455_solr ] o.a.s.c.Overseer Overseer (id=98625216274694147-127.0.0.1:55455_solr-n_0000000000) starting
[junit4] 2> 4481360 INFO (jetty-launcher-3991-thread-1) [n:127.0.0.1:55455_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55455_solr
[junit4] 2> 4481361 INFO (zkCallback-3998-thread-1-processing-n:127.0.0.1:55455_solr) [n:127.0.0.1:55455_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 4481415 INFO (jetty-launcher-3991-thread-1) [n:127.0.0.1:55455_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55455.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@ba3315f
[junit4] 2> 4481423 INFO (jetty-launcher-3991-thread-1) [n:127.0.0.1:55455_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55455.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@ba3315f
[junit4] 2> 4481423 INFO (jetty-launcher-3991-thread-1) [n:127.0.0.1:55455_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55455.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@ba3315f
[junit4] 2> 4481424 INFO (jetty-launcher-3991-thread-1) [n:127.0.0.1:55455_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderElectionContextKeyTest_EDB3304C08F978F6-001/tempDir-001/node1/.
[junit4] 2> 4481453 INFO (SUITE-LeaderElectionContextKeyTest-seed#[EDB3304C08F978F6]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 4481453 INFO (SUITE-LeaderElectionContextKeyTest-seed#[EDB3304C08F978F6]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:45185/solr ready
[junit4] 2> 4481456 INFO (qtp1092010652-89452) [n:127.0.0.1:55455_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=config&maxShardsPerNode=1000&name=testCollection1&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 4481465 INFO (OverseerThreadFactory-7932-thread-1-processing-n:127.0.0.1:55455_solr) [n:127.0.0.1:55455_solr ] o.a.s.c.CreateCollectionCmd Create collection testCollection1
[junit4] 2> 4481578 INFO (OverseerStateUpdate-98625216274694147-127.0.0.1:55455_solr-n_0000000000) [n:127.0.0.1:55455_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testCollection1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"testCollection1_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:55455/solr",
[junit4] 2> "type":"NRT"}
[junit4] 2> 4481583 INFO (OverseerStateUpdate-98625216274694147-127.0.0.1:55455_solr-n_0000000000) [n:127.0.0.1:55455_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testCollection1",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"testCollection1_shard2_replica_n3",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:55455/solr",
[junit4] 2> "type":"NRT"}
[junit4] 2> 4481690 INFO (qtp1092010652-89450) [n:127.0.0.1:55455_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=config&newCollection=true&name=testCollection1_shard1_replica_n1&action=CREATE&numShards=2&collection=testCollection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 4481691 INFO (qtp1092010652-89454) [n:127.0.0.1:55455_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=config&newCollection=true&name=testCollection1_shard2_replica_n3&action=CREATE&numShards=2&collection=testCollection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 4481691 INFO (qtp1092010652-89454) [n:127.0.0.1:55455_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 4481797 INFO (zkCallback-3998-thread-1-processing-n:127.0.0.1:55455_solr) [n:127.0.0.1:55455_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection1/state.json] for collection [testCollection1] has occurred - updating... (live nodes size: [1])
[junit4] 2> 4482756 INFO (qtp1092010652-89454) [n:127.0.0.1:55455_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 4482761 INFO (qtp1092010652-89450) [n:127.0.0.1:55455_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 4482768 INFO (qtp1092010652-89450) [n:127.0.0.1:55455_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.s.IndexSchema [testCollection1_shard1_replica_n1] Schema name=minimal
[junit4] 2> 4482772 INFO (qtp1092010652-89454) [n:127.0.0.1:55455_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.s.IndexSchema [testCollection1_shard2_replica_n3] Schema name=minimal
[junit4] 2> 4482774 INFO (qtp1092010652-89454) [n:127.0.0.1:55455_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 4482774 INFO (qtp1092010652-89454) [n:127.0.0.1:55455_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.c.CoreContainer Creating SolrCore 'testCollection1_shard2_replica_n3' using configuration from collection testCollection1, trusted=true
[junit4] 2> 4482777 INFO (qtp1092010652-89454) [n:127.0.0.1:55455_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55455.solr.core.testCollection1.shard2.replica_n3' (registry 'solr.core.testCollection1.shard2.replica_n3') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@ba3315f
[junit4] 2> 4482777 INFO (qtp1092010652-89454) [n:127.0.0.1:55455_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 4482777 INFO (qtp1092010652-89454) [n:127.0.0.1:55455_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.c.SolrCore [[testCollection1_shard2_replica_n3] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderElectionContextKeyTest_EDB3304C08F978F6-001/tempDir-001/node1/testCollection1_shard2_replica_n3], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderElectionContextKeyTest_EDB3304C08F978F6-001/tempDir-001/node1/./testCollection1_shard2_replica_n3/data/]
[junit4] 2> 4482780 INFO (qtp1092010652-89450) [n:127.0.0.1:55455_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 4482780 INFO (qtp1092010652-89450) [n:127.0.0.1:55455_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'testCollection1_shard1_replica_n1' using configuration from collection testCollection1, trusted=true
[junit4] 2> 4482783 INFO (qtp1092010652-89450) [n:127.0.0.1:55455_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55455.solr.core.testCollection1.shard1.replica_n1' (registry 'solr.core.testCollection1.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@ba3315f
[junit4] 2> 4482783 INFO (qtp1092010652-89450) [n:127.0.0.1:55455_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 4482783 INFO (qtp1092010652-89450) [n:127.0.0.1:55455_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.c.SolrCore [[testCollection1_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderElectionContextKeyTest_EDB3304C08F978F6-001/tempDir-001/node1/testCollection1_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderElectionContextKeyTest_EDB3304C08F978F6-001/tempDir-001/node1/./testCollection1_shard1_replica_n1/data/]
[junit4] 2> 4482896 INFO (qtp1092010652-89454) [n:127.0.0.1:55455_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 4482896 INFO (qtp1092010652-89454) [n:127.0.0.1:55455_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 4482897 INFO (qtp1092010652-89454) [n:127.0.0.1:55455_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 4482897 INFO (qtp1092010652-89454) [n:127.0.0.1:55455_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 4482899 INFO (qtp1092010652-89454) [n:127.0.0.1:55455_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.s.SolrIndexSearcher Opening [Searcher@5ce1d73d[testCollection1_shard2_replica_n3] main]
[junit4] 2> 4482900 INFO (qtp1092010652-89454) [n:127.0.0.1:55455_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/config
[junit4] 2> 4482900 INFO (qtp1092010652-89454) [n:127.0.0.1:55455_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/config
[junit4] 2> 4482901 INFO (qtp1092010652-89454) [n:127.0.0.1:55455_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 4482901 INFO (qtp1092010652-89454) [n:127.0.0.1:55455_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1578003462204948480
[junit4] 2> 4482907 INFO (searcherExecutor-7935-thread-1-processing-n:127.0.0.1:55455_solr x:testCollection1_shard2_replica_n3 s:shard2 c:testCollection1 r:core_node4) [n:127.0.0.1:55455_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.c.SolrCore [testCollection1_shard2_replica_n3] Registered new searcher Searcher@5ce1d73d[testCollection1_shard2_replica_n3] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 4482907 INFO (qtp1092010652-89454) [n:127.0.0.1:55455_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 4482907 INFO (qtp1092010652-89454) [n:127.0.0.1:55455_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 4482907 INFO (qtp1092010652-89454) [n:127.0.0.1:55455_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:55455/solr/testCollection1_shard2_replica_n3/
[junit4] 2> 4482907 INFO (qtp1092010652-89454) [n:127.0.0.1:55455_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 4482907 INFO (qtp1092010652-89454) [n:127.0.0.1:55455_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.c.SyncStrategy http://127.0.0.1:55455/solr/testCollection1_shard2_replica_n3/ has no replicas
[junit4] 2> 4482907 INFO (qtp1092010652-89454) [n:127.0.0.1:55455_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 4482910 INFO (qtp1092010652-89454) [n:127.0.0.1:55455_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:55455/solr/testCollection1_shard2_replica_n3/ shard2
[junit4] 2> 4482916 INFO (qtp1092010652-89450) [n:127.0.0.1:55455_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 4482916 INFO (qtp1092010652-89450) [n:127.0.0.1:55455_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 4482917 INFO (qtp1092010652-89450) [n:127.0.0.1:55455_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 4482917 INFO (qtp1092010652-89450) [n:127.0.0.1:55455_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 4482918 INFO (qtp1092010652-89450) [n:127.0.0.1:55455_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@21f33d09[testCollection1_shard1_replica_n1] main]
[junit4] 2> 4482919 INFO (qtp1092010652-89450) [n:127.0.0.1:55455_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/config
[junit4] 2> 4482919 INFO (qtp1092010652-89450) [n:127.0.0.1:55455_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/config
[junit4] 2> 4482920 INFO (qtp1092010652-89450) [n:127.0.0.1:55455_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 4482920 INFO (qtp1092010652-89450) [n:127.0.0.1:55455_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1578003462224871424
[junit4] 2> 4482923 INFO (searcherExecutor-7936-thread-1-processing-n:127.0.0.1:55455_solr x:testCollection1_shard1_replica_n1 s:shard1 c:testCollection1 r:core_node2) [n:127.0.0.1:55455_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.c.SolrCore [testCollection1_shard1_replica_n1] Registered new searcher Searcher@21f33d09[testCollection1_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 4482924 INFO (qtp1092010652-89450) [n:127.0.0.1:55455_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 4482924 INFO (qtp1092010652-89450) [n:127.0.0.1:55455_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 4482924 INFO (qtp1092010652-89450) [n:127.0.0.1:55455_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:55455/solr/testCollection1_shard1_replica_n1/
[junit4] 2> 4482924 INFO (qtp1092010652-89450) [n:127.0.0.1:55455_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 4482924 INFO (qtp1092010652-89450) [n:127.0.0.1:55455_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:55455/solr/testCollection1_shard1_replica_n1/ has no replicas
[junit4] 2> 4482924 INFO (qtp1092010652-89450) [n:127.0.0.1:55455_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 4482926 INFO (qtp1092010652-89450) [n:127.0.0.1:55455_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:55455/solr/testCollection1_shard1_replica_n1/ shard1
[junit4] 2> 4483086 INFO (zkCallback-3998-thread-1-processing-n:127.0.0.1:55455_solr) [n:127.0.0.1:55455_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection1/state.json] for collection [testCollection1] has occurred - updating... (live nodes size: [1])
[junit4] 2> 4483135 INFO (qtp1092010652-89450) [n:127.0.0.1:55455_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 4483135 INFO (qtp1092010652-89454) [n:127.0.0.1:55455_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 4483138 INFO (qtp1092010652-89450) [n:127.0.0.1:55455_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node2&collection.configName=config&newCollection=true&name=testCollection1_shard1_replica_n1&action=CREATE&numShards=2&collection=testCollection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1448
[junit4] 2> 4483139 INFO (qtp1092010652-89454) [n:127.0.0.1:55455_solr c:testCollection1 s:shard2 r:core_node4 x:testCollection1_shard2_replica_n3] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=config&newCollection=true&name=testCollection1_shard2_replica_n3&action=CREATE&numShards=2&collection=testCollection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1448
[junit4] 2> 4483143 INFO (qtp1092010652-89452) [n:127.0.0.1:55455_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 4483244 INFO (zkCallback-3998-thread-2-processing-n:127.0.0.1:55455_solr) [n:127.0.0.1:55455_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection1/state.json] for collection [testCollection1] has occurred - updating... (live nodes size: [1])
[junit4] 2> 4483465 INFO (OverseerCollectionConfigSetProcessor-98625216274694147-127.0.0.1:55455_solr-n_0000000000) [n:127.0.0.1:55455_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 4484144 INFO (qtp1092010652-89452) [n:127.0.0.1:55455_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=config&maxShardsPerNode=1000&name=testCollection1&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=2687
[junit4] 2> 4484144 INFO (qtp1092010652-89453) [n:127.0.0.1:55455_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=config&maxShardsPerNode=1000&name=testCollection2&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 4484156 INFO (OverseerThreadFactory-7932-thread-2-processing-n:127.0.0.1:55455_solr) [n:127.0.0.1:55455_solr ] o.a.s.c.CreateCollectionCmd Create collection testCollection2
[junit4] 2> 4484373 INFO (OverseerStateUpdate-98625216274694147-127.0.0.1:55455_solr-n_0000000000) [n:127.0.0.1:55455_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testCollection2",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"testCollection2_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:55455/solr",
[junit4] 2> "type":"NRT"}
[junit4] 2> 4484375 INFO (OverseerStateUpdate-98625216274694147-127.0.0.1:55455_solr-n_0000000000) [n:127.0.0.1:55455_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testCollection2",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"testCollection2_shard2_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:55455/solr",
[junit4] 2> "type":"NRT"}
[junit4] 2> 4484677 INFO (qtp1092010652-89455) [n:127.0.0.1:55455_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=config&newCollection=true&name=testCollection2_shard2_replica_n2&action=CREATE&numShards=2&collection=testCollection2&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 4484678 INFO (qtp1092010652-89449) [n:127.0.0.1:55455_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=config&newCollection=true&name=testCollection2_shard1_replica_n1&action=CREATE&numShards=2&collection=testCollection2&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 4484787 INFO (zkCallback-3998-thread-2-processing-n:127.0.0.1:55455_solr) [n:127.0.0.1:55455_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection2/state.json] for collection [testCollection2] has occurred - updating... (live nodes size: [1])
[junit4] 2> 4485704 INFO (qtp1092010652-89455) [n:127.0.0.1:55455_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 4485709 INFO (qtp1092010652-89449) [n:127.0.0.1:55455_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 4485754 INFO (qtp1092010652-89455) [n:127.0.0.1:55455_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.s.IndexSchema [testCollection2_shard2_replica_n2] Schema name=minimal
[junit4] 2> 4485757 INFO (qtp1092010652-89449) [n:127.0.0.1:55455_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.s.IndexSchema [testCollection2_shard1_replica_n1] Schema name=minimal
[junit4] 2> 4485759 INFO (qtp1092010652-89449) [n:127.0.0.1:55455_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 4485759 INFO (qtp1092010652-89449) [n:127.0.0.1:55455_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'testCollection2_shard1_replica_n1' using configuration from collection testCollection2, trusted=true
[junit4] 2> 4485759 INFO (qtp1092010652-89449) [n:127.0.0.1:55455_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55455.solr.core.testCollection2.shard1.replica_n1' (registry 'solr.core.testCollection2.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@ba3315f
[junit4] 2> 4485760 INFO (qtp1092010652-89449) [n:127.0.0.1:55455_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 4485760 INFO (qtp1092010652-89449) [n:127.0.0.1:55455_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.c.SolrCore [[testCollection2_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderElectionContextKeyTest_EDB3304C08F978F6-001/tempDir-001/node1/testCollection2_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderElectionContextKeyTest_EDB3304C08F978F6-001/tempDir-001/node1/./testCollection2_shard1_replica_n1/data/]
[junit4] 2> 4485762 INFO (qtp1092010652-89455) [n:127.0.0.1:55455_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 4485762 INFO (qtp1092010652-89455) [n:127.0.0.1:55455_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'testCollection2_shard2_replica_n2' using configuration from collection testCollection2, trusted=true
[junit4] 2> 4485763 INFO (qtp1092010652-89455) [n:127.0.0.1:55455_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55455.solr.core.testCollection2.shard2.replica_n2' (registry 'solr.core.testCollection2.shard2.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@ba3315f
[junit4] 2> 4485763 INFO (qtp1092010652-89455) [n:127.0.0.1:55455_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 4485763 INFO (qtp1092010652-89455) [n:127.0.0.1:55455_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.c.SolrCore [[testCollection2_shard2_replica_n2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderElectionContextKeyTest_EDB3304C08F978F6-001/tempDir-001/node1/testCollection2_shard2_replica_n2], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderElectionContextKeyTest_EDB3304C08F978F6-001/tempDir-001/node1/./testCollection2_shard2_replica_n2/data/]
[junit4] 2> 4485879 INFO (qtp1092010652-89455) [n:127.0.0.1:55455_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 4485879 INFO (qtp1092010652-89455) [n:127.0.0.1:55455_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 4485880 INFO (qtp1092010652-89455) [n:127.0.0.1:55455_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 4485880 INFO (qtp1092010652-89455) [n:127.0.0.1:55455_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 4485881 INFO (qtp1092010652-89455) [n:127.0.0.1:55455_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@3bdab1e[testCollection2_shard2_replica_n2] main]
[junit4] 2> 4485882 INFO (qtp1092010652-89455) [n:127.0.0.1:55455_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/config
[junit4] 2> 4485882 INFO (qtp1092010652-89455) [n:127.0.0.1:55455_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/config
[junit4] 2> 4485883 INFO (qtp1092010652-89455) [n:127.0.0.1:55455_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 4485883 INFO (qtp1092010652-89455) [n:127.0.0.1:55455_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1578003465331802112
[junit4] 2> 4485901 INFO (qtp1092010652-89455) [n:127.0.0.1:55455_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 4485901 INFO (qtp1092010652-89455) [n:127.0.0.1:55455_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 4485901 INFO (qtp1092010652-89455) [n:127.0.0.1:55455_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:55455/solr/testCollection2_shard2_replica_n2/
[junit4] 2> 4485901 INFO (qtp1092010652-89455) [n:127.0.0.1:55455_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 4485902 INFO (qtp1092010652-89455) [n:127.0.0.1:55455_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.c.SyncStrategy http://127.0.0.1:55455/solr/testCollection2_shard2_replica_n2/ has no replicas
[junit4] 2> 4485902 INFO (qtp1092010652-89455) [n:127.0.0.1:55455_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 4485903 INFO (searcherExecutor-7946-thread-1-processing-n:127.0.0.1:55455_solr x:testCollection2_shard2_replica_n2 s:shard2 c:testCollection2 r:core_node4) [n:127.0.0.1:55455_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.c.SolrCore [testCollection2_shard2_replica_n2] Registered new searcher Searcher@3bdab1e[testCollection2_shard2_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 4485910 INFO (qtp1092010652-89455) [n:127.0.0.1:55455_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:55455/solr/testCollection2_shard2_replica_n2/ shard2
[junit4] 2> 4485925 INFO (qtp1092010652-89449) [n:127.0.0.1:55455_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 4485925 INFO (qtp1092010652-89449) [n:127.0.0.1:55455_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 4485926 INFO (qtp1092010652-89449) [n:127.0.0.1:55455_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 4485926 INFO (qtp1092010652-89449) [n:127.0.0.1:55455_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 4485927 INFO (qtp1092010652-89449) [n:127.0.0.1:55455_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@59ad5686[testCollection2_shard1_replica_n1] main]
[junit4] 2> 4485928 INFO (qtp1092010652-89449) [n:127.0.0.1:55455_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/config
[junit4] 2> 4485928 INFO (qtp1092010652-89449) [n:127.0.0.1:55455_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/config
[junit4] 2> 4485928 INFO (qtp1092010652-89449) [n:127.0.0.1:55455_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 4485929 INFO (qtp1092010652-89449) [n:127.0.0.1:55455_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1578003465380036608
[junit4] 2> 4485952 INFO (qtp1092010652-89449) [n:127.0.0.1:55455_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 4485952 INFO (qtp1092010652-89449) [n:127.0.0.1:55455_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 4485952 INFO (qtp1092010652-89449) [n:127.0.0.1:55455_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:55455/solr/testCollection2_shard1_replica_n1/
[junit4] 2> 4485952 INFO (qtp1092010652-89449) [n:127.0.0.1:55455_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 4485952 INFO (qtp1092010652-89449) [n:127.0.0.1:55455_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:55455/solr/testCollection2_shard1_replica_n1/ has no replicas
[junit4] 2> 4485952 INFO (qtp1092010652-89449) [n:127.0.0.1:55455_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 4485955 INFO (qtp1092010652-89449) [n:127.0.0.1:55455_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:55455/solr/testCollection2_shard1_replica_n1/ shard1
[junit4] 2> 4485960 INFO (searcherExecutor-7945-thread-1-processing-n:127.0.0.1:55455_solr x:testCollection2_shard1_replica_n1 s:shard1 c:testCollection2 r:core_node3) [n:127.0.0.1:55455_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.c.SolrCore [testCollection2_shard1_replica_n1] Registered new searcher Searcher@59ad5686[testCollection2_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 4486086 INFO (zkCallback-3998-thread-2-processing-n:127.0.0.1:55455_solr) [n:127.0.0.1:55455_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection2/state.json] for collection [testCollection2] has occurred - updating... (live nodes size: [1])
[junit4] 2> 4486135 INFO (qtp1092010652-89449) [n:127.0.0.1:55455_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 4486135 INFO (qtp1092010652-89455) [n:127.0.0.1:55455_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 4486136 INFO (qtp1092010652-89455) [n:127.0.0.1:55455_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=config&newCollection=true&name=testCollection2_shard2_replica_n2&action=CREATE&numShards=2&collection=testCollection2&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1459
[junit4] 2> 4486137 INFO (qtp1092010652-89449) [n:127.0.0.1:55455_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=config&newCollection=true&name=testCollection2_shard1_replica_n1&action=CREATE&numShards=2&collection=testCollection2&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1459
[junit4] 2> 4486139 INFO (qtp1092010652-89453) [n:127.0.0.1:55455_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 4486157 INFO (OverseerCollectionConfigSetProcessor-98625216274694147-127.0.0.1:55455_solr-n_0000000000) [n:127.0.0.1:55455_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 4486240 INFO (zkCallback-3998-thread-2-processing-n:127.0.0.1:55455_solr) [n:127.0.0.1:55455_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testCollection2/state.json] for collection [testCollection2] has occurred - updating... (live nodes size: [1])
[junit4] 2> 4487139 INFO (qtp1092010652-89453) [n:127.0.0.1:55455_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=config&maxShardsPerNode=1000&name=testCollection2&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=2994
[junit4] 2> 4487139 INFO (SUITE-LeaderElectionContextKeyTest-seed#[EDB3304C08F978F6]-worker) [ ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: testCollection1 failOnTimeout:true timeout (sec):30
[junit4] 2> 4487140 INFO (SUITE-LeaderElectionContextKeyTest-seed#[EDB3304C08F978F6]-worker) [ ] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: testCollection1
[junit4] 2> 4487140 INFO (SUITE-LeaderElectionContextKeyTest-seed#[EDB3304C08F978F6]-worker) [ ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: testCollection2 failOnTimeout:true timeout (sec):30
[junit4] 2> 4487140 INFO (SUITE-LeaderElectionContextKeyTest-seed#[EDB3304C08F978F6]-worker) [ ] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: testCollection2
[junit4] 2> 4487210 INFO (TEST-LeaderElectionContextKeyTest.test-seed#[EDB3304C08F978F6]) [ ] o.a.s.SolrTestCaseJ4 ###Starting test
[junit4] 2> 4487217 INFO (qtp1092010652-89450) [n:127.0.0.1:55455_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testCollection1.shard2.replica_n3, tag=null
[junit4] 2> 4487217 INFO (qtp1092010652-89450) [n:127.0.0.1:55455_solr ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6d2f972a: rootName = solr_55455, domain = solr.core.testCollection1.shard2.replica_n3, service url = null, agent id = null] for registry solr.core.testCollection1.shard2.replica_n3 / com.codahale.metrics.MetricRegistry@7f17bf50
[junit4] 2> 4487233 INFO (qtp1092010652-89450) [n:127.0.0.1:55455_solr ] o.a.s.c.SolrCore [testCollection1_shard2_replica_n3] CLOSING SolrCore org.apache.solr.core.SolrCore@38d25fbf
[junit4] 2> 4487233 INFO (qtp1092010652-89450) [n:127.0.0.1:55455_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testCollection1.shard2.replica_n3, tag=953311167
[junit4] 2> 4487233 INFO (qtp1092010652-89450) [n:127.0.0.1:55455_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testCollection1.shard2.leader, tag=953311167
[junit4] 2> 4487247 INFO (qtp1092010652-89450) [n:127.0.0.1:55455_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=false&core=testCollection1_shard2_replica_n3&deleteDataDir=false&action=UNLOAD&indexInfo=true&wt=javabin&version=2&deleteIndex=false} status=0 QTime=30
[junit4] 2> 4487248 INFO (TEST-LeaderElectionContextKeyTest.test-seed#[EDB3304C08F978F6]) [ ] o.a.s.SolrTestCaseJ4 ###Ending test
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=LeaderElectionContextKeyTest -Dtests.method=test -Dtests.seed=EDB3304C08F978F6 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=et -Dtests.timezone=BET -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] FAILURE 0.13s J0 | LeaderElectionContextKeyTest.test <<<
[junit4] > Throwable #1: java.lang.AssertionError
[junit4] > at __randomizedtesting.SeedInfo.seed([EDB3304C08F978F6:65E70F96A605150E]:0)
[junit4] > at org.apache.solr.cloud.LeaderElectionContextKeyTest.test(LeaderElectionContextKeyTest.java:88)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 4487285 INFO (jetty-closer-3992-thread-1) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@5c4f747d{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 4487287 INFO (jetty-closer-3992-thread-1) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1316825863
[junit4] 2> 4487287 INFO (jetty-closer-3992-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 4487287 INFO (jetty-closer-3992-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@557e84cb: rootName = solr_55455, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@ac016d3
[junit4] 2> 4487292 INFO (jetty-closer-3992-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 4487292 INFO (jetty-closer-3992-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@60151eba: rootName = solr_55455, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@3e9674a7
[junit4] 2> 4487298 INFO (jetty-closer-3992-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 4487298 INFO (jetty-closer-3992-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@78ac5634: rootName = solr_55455, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@75c6f35e
[junit4] 2> 4487300 INFO (jetty-closer-3992-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 4487320 INFO (coreCloseExecutor-7955-thread-1) [n:127.0.0.1:55455_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.c.SolrCore [testCollection1_shard1_replica_n1] CLOSING SolrCore org.apache.solr.core.SolrCore@1b57f584
[junit4] 2> 4487320 INFO (coreCloseExecutor-7955-thread-1) [n:127.0.0.1:55455_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testCollection1.shard1.replica_n1, tag=458749316
[junit4] 2> 4487320 INFO (coreCloseExecutor-7955-thread-1) [n:127.0.0.1:55455_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@17c004db: rootName = solr_55455, domain = solr.core.testCollection1.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.testCollection1.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@550e8a
[junit4] 2> 4487326 INFO (coreCloseExecutor-7955-thread-2) [n:127.0.0.1:55455_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.c.SolrCore [testCollection2_shard2_replica_n2] CLOSING SolrCore org.apache.solr.core.SolrCore@26b1a970
[junit4] 2> 4487328 INFO (coreCloseExecutor-7955-thread-2) [n:127.0.0.1:55455_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testCollection2.shard2.replica_n2, tag=649177456
[junit4] 2> 4487329 INFO (coreCloseExecutor-7955-thread-2) [n:127.0.0.1:55455_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3df11ce8: rootName = solr_55455, domain = solr.core.testCollection2.shard2.replica_n2, service url = null, agent id = null] for registry solr.core.testCollection2.shard2.replica_n2 / com.codahale.metrics.MetricRegistry@27108e67
[junit4] 2> 4487333 INFO (coreCloseExecutor-7955-thread-3) [n:127.0.0.1:55455_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.c.SolrCore [testCollection2_shard1_replica_n1] CLOSING SolrCore org.apache.solr.core.SolrCore@16c06e15
[junit4] 2> 4487338 INFO (coreCloseExecutor-7955-thread-1) [n:127.0.0.1:55455_solr c:testCollection1 s:shard1 r:core_node2 x:testCollection1_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testCollection1.shard1.leader, tag=458749316
[junit4] 2> 4487338 INFO (coreCloseExecutor-7955-thread-3) [n:127.0.0.1:55455_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.testCollection2.shard1.replica_n1, tag=381709845
[junit4] 2> 4487338 INFO (coreCloseExecutor-7955-thread-3) [n:127.0.0.1:55455_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@399c9e82: rootName = solr_55455, domain = solr.core.testCollection2.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.testCollection2.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@7a55564c
[junit4] 2> 4487355 INFO (coreCloseExecutor-7955-thread-3) [n:127.0.0.1:55455_solr c:testCollection2 s:shard1 r:core_node3 x:testCollection2_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testCollection2.shard1.leader, tag=381709845
[junit4] 2> 4487364 INFO (coreCloseExecutor-7955-thread-2) [n:127.0.0.1:55455_solr c:testCollection2 s:shard2 r:core_node4 x:testCollection2_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testCollection2.shard2.leader, tag=649177456
[junit4] 2> 4487376 INFO (jetty-closer-3992-thread-1) [ ] o.a.s.c.Overseer Overseer (id=98625216274694147-127.0.0.1:55455_solr-n_0000000000) closing
[junit4] 2> 4487376 INFO (OverseerStateUpdate-98625216274694147-127.0.0.1:55455_solr-n_0000000000) [n:127.0.0.1:55455_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:55455_solr
[junit4] 2> 4487387 WARN (zkCallback-3998-thread-2-processing-n:127.0.0.1:55455_solr) [n:127.0.0.1:55455_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 4487387 INFO (jetty-closer-3992-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@c1675b2{/solr,null,UNAVAILABLE}
[junit4] 2> 4487388 ERROR (SUITE-LeaderElectionContextKeyTest-seed#[EDB3304C08F978F6]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 4487389 INFO (SUITE-LeaderElectionContextKeyTest-seed#[EDB3304C08F978F6]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:45185 45185
[junit4] 2> 4487478 INFO (Thread-73365) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:45185 45185
[junit4] 2> 4487480 WARN (Thread-73365) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/aliases.json
[junit4] 2> 2 /solr/clusterprops.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/clusterstate.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/live_nodes
[junit4] 2> 2 /solr/collections
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LeaderElectionContextKeyTest_EDB3304C08F978F6-001
[junit4] 2> Sep 08, 2017 8:09:37 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarity(queryNorm=true): {}, locale=et, timezone=BET
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=250667432,total=521666560
[junit4] 2> NOTE: All tests run in this JVM: [TestManagedSchemaThreadSafety, TestHashQParserPlugin, TestSolrDeletionPolicy1, TestSolrConfigHandler, TestNumericTokenStream, JSONWriterTest, DataDrivenBlockJoinTest, TestComplexPhraseLeadingWildcard, UpdateParamsTest, TestConfigSetProperties, TestCloudSearcherWarming, TestEmbeddedSolrServerAdminHandler, TestReplicationHandlerBackup, TestIBSimilarityFactory, RemoteQueryErrorTest, TestRandomRequestDistribution, TestWordDelimiterFilterFactory, TestExpandComponent, HdfsLockFactoryTest, WordBreakSolrSpellCheckerTest, TestPayloadScoreQParserPlugin, BitVectorTest, ReplicaListTransformerTest, HdfsChaosMonkeySafeLeaderTest, HLLSerializationTest, V2StandaloneTest, TestClusterStateMutator, SynonymTokenizerTest, TestReloadAndDeleteDocs, TestInPlaceUpdatesDistrib, TestDistribDocBasedVersion, TestHighlightDedupGrouping, TestSchemaVersionResource, TestStressCloudBlindAtomicUpdates, CollectionReloadTest, TestJettySolrRunner, TestJsonFacetRefinement, TestLazyCores, TestConfigSetsAPIZkFailure, TestCustomStream, TestDistributedSearch, SyncSliceTest, TestIndexingPerformance, TestSQLHandler, TestCSVLoader, SharedFSAutoReplicaFailoverUtilsTest, SuggesterWFSTTest, EnumFieldTest, LargeFieldTest, BasicZkTest, MissingSegmentRecoveryTest, TestSubQueryTransformerDistrib, TestRemoteStreaming, TestLeaderInitiatedRecoveryThread, TestCSVResponseWriter, TestJoin, TestCloudPivotFacet, StatsReloadRaceTest, SolrMetricReporterTest, ConfigSetsAPITest, TestFaceting, TestNoOpRegenerator, DocumentAnalysisRequestHandlerTest, DocValuesNotIndexedTest, TestSolrCLIRunExample, TestSolrQueryParser, SolrGraphiteReporterTest, SuggestComponentContextFilterQueryTest, TestCrossCoreJoin, TestSolrCoreSnapshots, FieldMutatingUpdateProcessorTest, MetricsConfigTest, TestStreamBody, HdfsBasicDistributedZk2Test, IndexBasedSpellCheckerTest, HighlighterConfigTest, TestMiniSolrCloudClusterSSL, SaslZkACLProviderTest, VersionInfoTest, TestAnalyzeInfixSuggestions, TestNumericTerms32, TestSolr4Spatial, DeleteStatusTest, BlobRepositoryCloudTest, TestOmitPositions, BinaryUpdateRequestHandlerTest, ExternalFileFieldSortTest, TestTrieFacet, DistributedFacetPivotWhiteBoxTest, LeaderElectionIntegrationTest, TestSolrCloudWithHadoopAuthPlugin, AsyncCallRequestStatusResponseTest, CollectionsAPIAsyncDistributedZkTest, TestSolrCloudWithDelegationTokens, TestDocumentBuilder, EchoParamsTest, TestFieldCacheReopen, QueryElevationComponentTest, TestFieldCacheVsDocValues, CurrencyFieldTypeTest, DebugComponentTest, TestSchemaNameResource, TestExactSharedStatsCache, AnalyticsMergeStrategyTest, TestSha256AuthenticationProvider, TestBinaryField, SpellingQueryConverterTest, ImplicitSnitchTest, TestFreeTextSuggestions, TestGraphMLResponseWriter, DeleteNodeTest, TestCollationField, ZkNodePropsTest, SolrCloudExampleTest, ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test, OpenCloseCoreStressTest, TestReplicationHandler, SolrCoreTest, PeerSyncTest, ConvertedLegacyTest, TestFunctionQuery, TestBadConfig, HighlighterTest, SoftAutoCommitTest, DistributedQueryElevationComponentTest, SolrIndexSplitterTest, SimplePostToolTest, TestCoreDiscovery, AnalysisAfterCoreReloadTest, TestExtendedDismaxParser, SpellCheckCollatorTest, CoreAdminHandlerTest, SolrRequestParserTest, TestFoldingMultitermQuery, TestStressLucene, TestTrie, SpatialFilterTest, PolyFieldTest, NoCacheHeaderTest, TestPseudoReturnFields, TestUpdate, TestAtomicUpdateErrorCases, DefaultValueUpdateProcessorTest, StatelessScriptUpdateProcessorFactoryTest, SortByFunctionTest, DocValuesMultiTest, SolrInfoBeanTest, XsltUpdateRequestHandlerTest, TestSurroundQueryParser, LukeRequestHandlerTest, TestWriterPerf, TestReversedWildcardFilterFactory, TestQueryTypes, FileBasedSpellCheckerTest, XmlUpdateRequestHandlerTest, RequestHandlersTest, DocumentBuilderTest, FieldAnalysisRequestHandlerTest, LoggingHandlerTest, RegexBoostProcessorTest, SolrPluginUtilsTest, IndexSchemaTest, MBeansHandlerTest, UniqFieldsUpdateProcessorFactoryTest, TestPHPSerializedResponseWriter, PingRequestHandlerTest, TestComponentsName, TestLFUCache, PrimUtilsTest, TestSuggestSpellingConverter, DOMUtilTest, RAMDirectoryFactoryTest, ClusterStateTest, SystemInfoHandlerTest, UUIDFieldTest, FileUtilsTest, CircularListTest, CursorPagingTest, SolrTestCaseJ4Test, TestSimpleTrackingShardHandler, TestEmbeddedSolrServerConstructors, TestEmbeddedSolrServerSchemaAPI, ActionThrottleTest, AliasIntegrationTest, CdcrBootstrapTest, CdcrVersionReplicationTest, CollectionStateFormat2Test, ConcurrentDeleteAndCreateCollectionTest, DeleteLastCustomShardedReplicaTest, DeleteShardTest, DistribCursorPagingTest, LeaderElectionContextKeyTest]
[junit4] Completed [504/731 (1!)] on J0 in 6.90s, 1 test, 1 failure <<< FAILURES!
[...truncated 11 lines...]
[junit4] Suite: org.apache.solr.cloud.HttpPartitionTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.HttpPartitionTest_EDB3304C08F978F6-001/init-core-data-001
[junit4] 2> 4481658 WARN (SUITE-HttpPartitionTest-seed#[EDB3304C08F978F6]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=14 numCloses=14
[junit4] 2> 4481659 INFO (SUITE-HttpPartitionTest-seed#[EDB3304C08F978F6]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 4481661 INFO (SUITE-HttpPartitionTest-seed#[EDB3304C08F978F6]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
[junit4] 2> 4481661 INFO (SUITE-HttpPartitionTest-seed#[EDB3304C08F978F6]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 4481661 INFO (SUITE-HttpPartitionTest-seed#[EDB3304C08F978F6]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 2> 4481664 INFO (TEST-HttpPartitionTest.test-seed#[EDB3304C08F978F6]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 4481664 INFO (Thread-18313) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 4481664 INFO (Thread-18313) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 4481666 ERROR (Thread-18313) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 4481764 INFO (TEST-HttpPartitionTest.test-seed#[EDB3304C08F978F6]) [ ] o.a.s.c.ZkTestServer start zk server on port:58750
[junit4] 2> 4481786 INFO (TEST-HttpPartitionTest.test-seed#[EDB3304C08F978F6]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 4481787 INFO (TEST-HttpPartitionTest.test-seed#[EDB3304C08F978F6]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 4481788 INFO (TEST-HttpPartitionTest.test-seed#[EDB3304C08F978F6]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 4481789 INFO (TEST-HttpPartitionTest.test-seed#[EDB3304C08F978F6]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 4481790 INFO (TEST-HttpPartitionTest.test-seed#[EDB3304C08F978F6]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 4481791 INFO (TEST-HttpPartitionTest.test-seed#[EDB3304C08F978F6]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 4481791 INFO (TEST-HttpPartitionTest.test-seed#[EDB3304C08F978F6]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 4481792 INFO (TEST-HttpPartitionTest.test-seed#[EDB3304C08F978F6]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 4481793 INFO (TEST-HttpPartitionTest.test-seed#[EDB3304C08F978F6]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 4481794 INFO (TEST-HttpPartitionTest.test-seed#[EDB3304C08F978F6]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 4481795 INFO (TEST-HttpPartitionTest.test-seed#[EDB3304C08F978F6]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 4481796 INFO (TEST-HttpPartitionTest.test-seed#[EDB3304C08F978F6]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use TLOG replicas unless explicitly asked otherwise
[junit4] 2> 4481953 INFO (TEST-HttpPartitionTest.test-seed#[EDB3304C08F978F6]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 4481963 INFO (TEST-HttpPartitionTest.test-seed#[EDB3304C08F978F6]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@69fef054{/,null,AVAILABLE}
[junit4] 2> 4481972 INFO (TEST-HttpPartitionTest.test-seed#[EDB3304C08F978F6]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@47623057{HTTP/1.1,[http/1.1]}{127.0.0.1:44562}
[junit4] 2> 4481972 INFO (TEST-HttpPartitionTest.test-seed#[EDB3304C08F978F6]) [ ] o.e.j.s.Server Started @4486951ms
[junit4] 2> 4481972 INFO (TEST-HttpPartitionTest.test-seed#[EDB3304C08F978F6]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.HttpPartitionTest_EDB3304C08F978F6-001/tempDir-001/control/data, hostContext=/, hostPort=43986, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.HttpPartitionTest_EDB3304C08F978F6-001/control-001/cores}
[junit4] 2> 4481972 ERROR (TEST-HttpPartitionTest.test-seed#[EDB3304C08F978F6]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 4481972 INFO (TEST-HttpPartitionTest.test-seed#[EDB3304C08F978F6]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.1.0
[junit4] 2> 4481972 INFO (TEST-HttpPartitionTest.test-seed#[EDB3304C08F978F6]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 4481972 INFO (TEST-HttpPartitionTest.test-seed#[EDB3304C08F978F6]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 4481972 INFO (TEST-HttpPartitionTest.test-seed#[EDB3304C08F978F6]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-09-08T20:09:28.703Z
[junit4] 2> 4481998 INFO (TEST-HttpPartitionTest.test-seed#[EDB3304C08F978F6]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 4481998 INFO (TEST-HttpPartitionTest.test-seed#[EDB3304C08F978F6]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.HttpPartitionTest_EDB3304C08F978F6-001/control-001/solr.xml
[junit4] 2> 4482005 INFO (TEST-HttpPartitionTest.test-seed#[EDB3304C08F978F6]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@29f1461d, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 4482029 INFO (TEST-HttpPartitionTest.test-seed#[EDB3304C08F978F6]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:58750/solr
[junit4] 2> 4482206 INFO (TEST-HttpPartitionTest.test-seed#[EDB3304C08F978F6]) [n:127.0.0.1:43986_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 4482207 INFO (TEST-HttpPartitionTest.test-seed#[EDB3304C08F978F6]) [n:127.0.0.1:43986_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:43986_
[junit4] 2> 4482208 INFO (TEST-HttpPartitionTest.test-seed#[EDB3304C08F978F6]) [n:127.0.0.1:43986_ ] o.a.s.c.Overseer Overseer (id=98625216079200260-127.0.0.1:43986_-n_0000000000) starting
[junit4] 2> 4482230 INFO (TEST-HttpPartitionTest.test-seed#[EDB3304C08F978F6]) [n:127.0.0.1:43986_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43986_
[junit4] 2> 4482231 INFO (zkCallback-30745-thread-1-processing-n:127.0.0.1:43986_) [n:127.0.0.1:43986_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 4482663 INFO (TEST-HttpPartitionTest.test-seed#[EDB3304C08F978F6]) [n:127.0.0.1:43986_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@29f1461d
[junit4] 2> 4482701 INFO (TEST-HttpPartitionTest.test-seed#[EDB3304C08F978F6]) [n:127.0.0.1:43986_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@29f1461d
[junit4] 2> 4482701 INFO (TEST-HttpPartitionTest.test-see
[...truncated too long message...]
itArgsOrSysPropConfig -Dtests.seed=EDB3304C08F978F6 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=hu-HU -Dtests.timezone=Asia/Almaty -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 3.23s J1 | HdfsDirectoryFactoryTest.testInitArgsOrSysPropConfig <<<
[junit4] > Throwable #1: java.lang.RuntimeException: The max direct memory is likely too low. Either increase it (by adding -XX:MaxDirectMemorySize=<size>g -XX:+UseLargePages to your containers startup args) or disable direct allocation using solr.hdfs.blockcache.direct.memory.allocation=false in solrconfig.xml. If you are putting the block cache on the heap, your java heap size might not be large enough. Failed allocating ~134.217728 MB.
[junit4] > at __randomizedtesting.SeedInfo.seed([EDB3304C08F978F6:1A1CF967F57092DD]:0)
[junit4] > at org.apache.solr.core.HdfsDirectoryFactory.createBlockCache(HdfsDirectoryFactory.java:311)
[junit4] > at org.apache.solr.core.HdfsDirectoryFactory.getBlockDirectoryCache(HdfsDirectoryFactory.java:287)
[junit4] > at org.apache.solr.core.HdfsDirectoryFactory.create(HdfsDirectoryFactory.java:227)
[junit4] > at org.apache.solr.core.HdfsDirectoryFactoryTest.testInitArgsOrSysPropConfig(HdfsDirectoryFactoryTest.java:111)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > Caused by: java.lang.OutOfMemoryError: Direct buffer memory
[junit4] > at java.nio.Bits.reserveMemory(Bits.java:694)
[junit4] > at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
[junit4] > at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
[junit4] > at org.apache.solr.store.blockcache.BlockCache.<init>(BlockCache.java:71)
[junit4] > at org.apache.solr.core.HdfsDirectoryFactory.createBlockCache(HdfsDirectoryFactory.java:309)
[junit4] > ... 42 more
[junit4] 2> 6019276 WARN (SUITE-HdfsDirectoryFactoryTest-seed#[EDB3304C08F978F6]-worker) [ ] o.a.h.h.s.d.DirectoryScanner DirectoryScanner: shutdown has been called
[junit4] 2> 6019502 INFO (SUITE-HdfsDirectoryFactoryTest-seed#[EDB3304C08F978F6]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
[junit4] 2> 6019603 WARN (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.core.HdfsDirectoryFactoryTest_EDB3304C08F978F6-001/tempDir-001/hdfsBaseDir/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.core.HdfsDirectoryFactoryTest_EDB3304C08F978F6-001/tempDir-001/hdfsBaseDir/data/data4/]] heartbeating to localhost/127.0.0.1:55357) [ ] o.a.h.h.s.d.IncrementalBlockReportManager IncrementalBlockReportManager interrupted
[junit4] 2> 6019603 WARN (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.core.HdfsDirectoryFactoryTest_EDB3304C08F978F6-001/tempDir-001/hdfsBaseDir/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.core.HdfsDirectoryFactoryTest_EDB3304C08F978F6-001/tempDir-001/hdfsBaseDir/data/data4/]] heartbeating to localhost/127.0.0.1:55357) [ ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-1217419194-10.41.0.5-1504902900480 (Datanode Uuid bb35aa8e-3d9c-4d5e-864b-4c6e35193a54) service to localhost/127.0.0.1:55357
[junit4] 2> 6019605 WARN (SUITE-HdfsDirectoryFactoryTest-seed#[EDB3304C08F978F6]-worker) [ ] o.a.h.h.s.d.DirectoryScanner DirectoryScanner: shutdown has been called
[junit4] 2> 6019731 INFO (SUITE-HdfsDirectoryFactoryTest-seed#[EDB3304C08F978F6]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
[junit4] 2> 6019920 WARN (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.core.HdfsDirectoryFactoryTest_EDB3304C08F978F6-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.core.HdfsDirectoryFactoryTest_EDB3304C08F978F6-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to localhost/127.0.0.1:55357) [ ] o.a.h.h.s.d.IncrementalBlockReportManager IncrementalBlockReportManager interrupted
[junit4] 2> 6019920 WARN (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.core.HdfsDirectoryFactoryTest_EDB3304C08F978F6-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.core.HdfsDirectoryFactoryTest_EDB3304C08F978F6-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to localhost/127.0.0.1:55357) [ ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-1217419194-10.41.0.5-1504902900480 (Datanode Uuid 073bec27-470d-4d1b-97c9-0e56d98182ad) service to localhost/127.0.0.1:55357
[junit4] 2> 6019945 INFO (SUITE-HdfsDirectoryFactoryTest-seed#[EDB3304C08F978F6]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.core.HdfsDirectoryFactoryTest_EDB3304C08F978F6-001
[junit4] 2> Sep 08, 2017 8:35:07 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 34 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarity(queryNorm=false): {}, locale=hu-HU, timezone=Asia/Almaty
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=2,free=278032104,total=525860864
[junit4] 2> NOTE: All tests run in this JVM: [TestAddFieldRealTimeGet, TestRTGBase, TestXIncludeConfig, TestSolrXml, SolrCoreMetricManagerTest, TestSubQueryTransformer, SolrCmdDistributorTest, TestLRUCache, CloneFieldUpdateProcessorFactoryTest, TestConfigSets, TestConfigSetsAPI, AssignBackwardCompatibilityTest, TestSystemCollAutoCreate, TestDefaultStatsCache, ShardRoutingTest, TestRebalanceLeaders, ConnectionReuseTest, TermVectorComponentTest, TestFieldCacheWithThreads, TestBlendedInfixSuggestions, TestSearcherReuse, BigEndianAscendingWordDeserializerTest, SpellCheckComponentTest, AutoScalingHandlerTest, ExplicitHLLTest, InfoHandlerTest, HdfsThreadLeakTest, TestUseDocValuesAsStored, TestSolrIndexConfig, AddSchemaFieldsUpdateProcessorFactoryTest, TestHashPartitioner, TestCursorMarkWithoutUniqueKey, TestSolrJ, IndexSchemaRuntimeFieldTest, TestStressInPlaceUpdates, ConnectionManagerTest, BaseCdcrDistributedZkTest, BlockJoinFacetRandomTest, TestDynamicLoading, TestDownShardTolerantSearch, TestCoreContainer, DistributedIntervalFacetingTest, CheckHdfsIndexTest, TestCloudInspectUtil, ClassificationUpdateProcessorFactoryTest, CoreAdminCreateDiscoverTest, TestPointFields, TestReqParamsAPI, SliceStateTest, TestFuzzyAnalyzedSuggestions, SpatialRPTFieldTypeTest, TestFiltering, CleanupOldIndexTest, OverseerRolesTest, TestManagedSynonymFilterFactory, TestExportWriter, TestLocalFSCloudBackupRestore, TestManagedResource, TestDistributedGrouping, TestClusterProperties, TestApiFramework, SolrJmxReporterCloudTest, CollectionsAPISolrJTest, DeleteReplicaTest, SmileWriterTest, TestSearchPerf, MoreLikeThisHandlerTest, LeaderInitiatedRecoveryOnShardRestartTest, TestSystemIdResolver, TestJsonFacets, SolrCloudReportersTest, SuggestComponentTest, DisMaxRequestHandlerTest, CSVRequestHandlerTest, TestPartialUpdateDeduplication, SolrMetricsIntegrationTest, TestJavabinTupleStreamParser, ZkSolrClientTest, DirectSolrSpellCheckerTest, JavabinLoaderTest, TestOnReconnectListenerSupport, SimpleCollectionCreateDeleteTest, ParsingFieldUpdateProcessorsTest, TestHighFrequencyDictionaryFactory, PeerSyncWithIndexFingerprintCachingTest, TestValueSourceCache, ForceLeaderTest, ZkStateReaderTest, TestComplexPhraseQParserPlugin, JsonLoaderTest, TestCloudSchemaless, DistanceFunctionTest, DistributedMLTComponentTest, UpdateRequestProcessorFactoryTest, MultiThreadedOCPTest, DirectSolrConnectionTest, BasicAuthStandaloneTest, ChaosMonkeyNothingIsSafeWithPullReplicasTest, DeleteInactiveReplicaTest, TestSolrCloudWithKerberosAlt, SampleTest, BasicFunctionalityTest, TestNumericTerms64, ShufflingReplicaListTransformerTest, AnalysisErrorHandlingTest, SchemaVersionSpecificBehaviorTest, BadComponentTest, DocValuesTest, TestPKIAuthenticationPlugin, TestFilteredDocIdSet, CacheHeaderTest, AnalyticsQueryTest, SuggesterTSTTest, HdfsWriteToMultipleCollectionsTest, BufferStoreTest, TestDynamicFieldResource, DistributedQueueTest, TestLegacyNumericUtils, QueryParsingTest, ZkControllerTest, SolrXmlInZkTest, BasicDistributedZkTest, LeaderFailureAfterFreshStartTest, LeaderInitiatedRecoveryOnCommitTest, OutOfBoxZkACLAndCredentialsProvidersTest, OverriddenZkACLAndCredentialsProvidersTest, OverseerModifyCollectionTest, OverseerTaskQueueTest, PeerSyncReplicationTest, ShardSplitTest, TestCloudRecovery, TestExclusionRuleCollectionAccess, TestLeaderElectionWithEmptyReplica, TestLeaderElectionZkExpiry, TestRequestForwarding, TestRequestStatusCollectionAPI, TestShortCircuitedRequests, TestTlogReplica, ZkFailoverTest, HdfsBasicDistributedZkTest, HdfsUnloadDistributedZkTest, ConfigureRecoveryStrategyTest, CoreSorterTest, DirectoryFactoryTest, HdfsDirectoryFactoryTest]
[junit4] Completed [572/731 (4!)] on J1 in 16.81s, 3 tests, 1 error <<< FAILURES!
[...truncated 46693 lines...]
[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 40 - Still
unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/40/
107 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.analytics.NoFacetCloudTest
Error Message:
Error starting up MiniSolrCloudCluster
Stack Trace:
java.lang.Exception: Error starting up MiniSolrCloudCluster
at __randomizedtesting.SeedInfo.seed([5ECF0772F96CC9BF]:0)
at org.apache.solr.cloud.MiniSolrCloudCluster.checkForExceptions(MiniSolrCloudCluster.java:507)
at org.apache.solr.cloud.MiniSolrCloudCluster.<init>(MiniSolrCloudCluster.java:251)
at org.apache.solr.cloud.SolrCloudTestCase$Builder.configure(SolrCloudTestCase.java:190)
at org.apache.solr.analytics.AbstractAnalyticsStatsCloudTest.setupCluster(AbstractAnalyticsStatsCloudTest.java:73)
at org.apache.solr.analytics.NoFacetCloudTest.populate(NoFacetCloudTest.java:62)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:874)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Suppressed: java.lang.RuntimeException: java.lang.NullPointerException
at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1600)
at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1659)
at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1316)
at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1145)
at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)
at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:384)
at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:247)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
Caused by: java.lang.NullPointerException
at org.apache.solr.util.configuration.providers.EnvSSLCredentialProvider.getCredential(EnvSSLCredentialProvider.java:57)
at org.apache.solr.util.configuration.providers.AbstractSSLCredentialProvider.getCredential(AbstractSSLCredentialProvider.java:53)
at org.apache.solr.util.configuration.SSLConfigurations.getPassword(SSLConfigurations.java:123)
at org.apache.solr.util.configuration.SSLConfigurations.getClientKeyStorePassword(SSLConfigurations.java:109)
at org.apache.solr.util.configuration.SSLConfigurations.init(SSLConfigurations.java:62)
at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:148)
at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139)
at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)
at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1596)
... 16 more
Suppressed: java.lang.RuntimeException: java.lang.NullPointerException
at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1600)
at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1659)
at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1316)
at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1145)
at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)
at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:384)
at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:247)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
Caused by: java.lang.NullPointerException
at org.apache.solr.util.configuration.providers.EnvSSLCredentialProvider.getCredential(EnvSSLCredentialProvider.java:57)
at org.apache.solr.util.configuration.providers.AbstractSSLCredentialProvider.getCredential(AbstractSSLCredentialProvider.java:53)
at org.apache.solr.util.configuration.SSLConfigurations.getPassword(SSLConfigurations.java:123)
at org.apache.solr.util.configuration.SSLConfigurations.getClientKeyStorePassword(SSLConfigurations.java:109)
at org.apache.solr.util.configuration.SSLConfigurations.init(SSLConfigurations.java:62)
at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:148)
at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139)
at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)
at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1596)
... 16 more
Suppressed: java.lang.RuntimeException: java.lang.NullPointerException
at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1600)
at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1659)
at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1316)
at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1145)
at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)
at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:384)
at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:247)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
Caused by: java.lang.NullPointerException
at org.apache.solr.util.configuration.providers.EnvSSLCredentialProvider.getCredential(EnvSSLCredentialProvider.java:57)
at org.apache.solr.util.configuration.providers.AbstractSSLCredentialProvider.getCredential(AbstractSSLCredentialProvider.java:53)
at org.apache.solr.util.configuration.SSLConfigurations.getPassword(SSLConfigurations.java:123)
at org.apache.solr.util.configuration.SSLConfigurations.getClientKeyStorePassword(SSLConfigurations.java:109)
at org.apache.solr.util.configuration.SSLConfigurations.init(SSLConfigurations.java:62)
at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:148)
at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139)
at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)
at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1596)
... 16 more
Suppressed: java.lang.RuntimeException: java.lang.NullPointerException
at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1600)
at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1659)
at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1316)
at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1145)
at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)
at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:384)
at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:247)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
Caused by: java.lang.NullPointerException
at org.apache.solr.util.configuration.providers.EnvSSLCredentialProvider.getCredential(EnvSSLCredentialProvider.java:57)
at org.apache.solr.util.configuration.providers.AbstractSSLCredentialProvider.getCredential(AbstractSSLCredentialProvider.java:53)
at org.apache.solr.util.configuration.SSLConfigurations.getPassword(SSLConfigurations.java:123)
at org.apache.solr.util.configuration.SSLConfigurations.getClientKeyStorePassword(SSLConfigurations.java:109)
at org.apache.solr.util.configuration.SSLConfigurations.init(SSLConfigurations.java:62)
at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:148)
at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139)
at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)
at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1596)
... 16 more
FAILED: junit.framework.TestSuite.org.apache.solr.analytics.NoFacetCloudTest
Error Message:
36 threads leaked from SUITE scope at org.apache.solr.analytics.NoFacetCloudTest: 1) Thread[id=52, name=qtp1795747797-52, state=RUNNABLE, group=TGRP-NoFacetCloudTest] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100) at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=85, name=qtp1317280112-85, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=74, name=qtp1317280112-74, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=55, name=qtp1795747797-55-acceptor-0@2c4e16a0-ServerConnector@2c430789{SSL,[ssl, http/1.1]}{127.0.0.1:52735}, state=RUNNABLE, group=TGRP-NoFacetCloudTest] at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:371) at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=62, name=qtp17719963-62, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 6) Thread[id=76, name=qtp1795747797-76, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 7) Thread[id=61, name=qtp17719963-61, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 8) Thread[id=66, name=qtp236770607-66, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 9) Thread[id=57, name=qtp17719963-57, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 10) Thread[id=70, name=qtp236770607-70, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 11) Thread[id=65, name=qtp236770607-65-acceptor-0@72fbe786-ServerConnector@11b04ca0{SSL,[ssl, http/1.1]}{127.0.0.1:56447}, state=RUNNABLE, group=TGRP-NoFacetCloudTest] at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:371) at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) 12) Thread[id=86, name=org.eclipse.jetty.server.session.HashSessionManager@2523fec9Timer, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 13) Thread[id=83, name=qtp1317280112-83, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 14) Thread[id=73, name=qtp1317280112-73-acceptor-0@62eb34bc-ServerConnector@55b44a27{SSL,[ssl, http/1.1]}{127.0.0.1:57546}, state=RUNNABLE, group=TGRP-NoFacetCloudTest] at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:371) at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) 15) Thread[id=54, name=qtp1795747797-54, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 16) Thread[id=72, name=qtp1317280112-72, state=RUNNABLE, group=TGRP-NoFacetCloudTest] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100) at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) 17) Thread[id=81, name=org.eclipse.jetty.server.session.HashSessionManager@7dfba6bTimer, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 18) Thread[id=71, name=qtp1317280112-71, state=RUNNABLE, group=TGRP-NoFacetCloudTest] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100) at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) 19) Thread[id=60, name=qtp17719963-60-acceptor-0@75324d2b-ServerConnector@c82087b{SSL,[ssl, http/1.1]}{127.0.0.1:43815}, state=RUNNABLE, group=TGRP-NoFacetCloudTest] at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:371) at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) 20) Thread[id=80, name=org.eclipse.jetty.server.session.HashSessionManager@4065ec0cTimer, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 21) Thread[id=63, name=qtp236770607-63, state=RUNNABLE, group=TGRP-NoFacetCloudTest] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100) at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) 22) Thread[id=69, name=qtp236770607-69, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 23) Thread[id=51, name=qtp17719963-51, state=RUNNABLE, group=TGRP-NoFacetCloudTest] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100) at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) 24) Thread[id=58, name=qtp17719963-58, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 25) Thread[id=67, name=qtp236770607-67, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 26) Thread[id=53, name=qtp1795747797-53, state=RUNNABLE, group=TGRP-NoFacetCloudTest] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100) at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) 27) Thread[id=84, name=qtp1317280112-84, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 28) Thread[id=56, name=qtp17719963-56, state=RUNNABLE, group=TGRP-NoFacetCloudTest] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100) at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) 29) Thread[id=64, name=qtp236770607-64, state=RUNNABLE, group=TGRP-NoFacetCloudTest] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100) at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) 30) Thread[id=79, name=qtp1795747797-79, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 31) Thread[id=75, name=qtp1317280112-75, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 32) Thread[id=68, name=qtp236770607-68, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 33) Thread[id=78, name=qtp1795747797-78, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 34) Thread[id=59, name=qtp17719963-59, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 35) Thread[id=77, name=qtp1795747797-77, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 36) Thread[id=82, name=org.eclipse.jetty.server.session.HashSessionManager@21fccc9bTimer, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 36 threads leaked from SUITE scope at org.apache.solr.analytics.NoFacetCloudTest:
1) Thread[id=52, name=qtp1795747797-52, state=RUNNABLE, group=TGRP-NoFacetCloudTest]
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)
at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=85, name=qtp1317280112-85, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=74, name=qtp1317280112-74, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
4) Thread[id=55, name=qtp1795747797-55-acceptor-0@2c4e16a0-ServerConnector@2c430789{SSL,[ssl, http/1.1]}{127.0.0.1:52735}, state=RUNNABLE, group=TGRP-NoFacetCloudTest]
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:371)
at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
5) Thread[id=62, name=qtp17719963-62, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
6) Thread[id=76, name=qtp1795747797-76, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
7) Thread[id=61, name=qtp17719963-61, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
8) Thread[id=66, name=qtp236770607-66, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
9) Thread[id=57, name=qtp17719963-57, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
10) Thread[id=70, name=qtp236770607-70, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
11) Thread[id=65, name=qtp236770607-65-acceptor-0@72fbe786-ServerConnector@11b04ca0{SSL,[ssl, http/1.1]}{127.0.0.1:56447}, state=RUNNABLE, group=TGRP-NoFacetCloudTest]
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:371)
at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
12) Thread[id=86, name=org.eclipse.jetty.server.session.HashSessionManager@2523fec9Timer, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
13) Thread[id=83, name=qtp1317280112-83, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
14) Thread[id=73, name=qtp1317280112-73-acceptor-0@62eb34bc-ServerConnector@55b44a27{SSL,[ssl, http/1.1]}{127.0.0.1:57546}, state=RUNNABLE, group=TGRP-NoFacetCloudTest]
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:371)
at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
15) Thread[id=54, name=qtp1795747797-54, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
16) Thread[id=72, name=qtp1317280112-72, state=RUNNABLE, group=TGRP-NoFacetCloudTest]
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)
at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
17) Thread[id=81, name=org.eclipse.jetty.server.session.HashSessionManager@7dfba6bTimer, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
18) Thread[id=71, name=qtp1317280112-71, state=RUNNABLE, group=TGRP-NoFacetCloudTest]
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)
at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
19) Thread[id=60, name=qtp17719963-60-acceptor-0@75324d2b-ServerConnector@c82087b{SSL,[ssl, http/1.1]}{127.0.0.1:43815}, state=RUNNABLE, group=TGRP-NoFacetCloudTest]
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:371)
at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
20) Thread[id=80, name=org.eclipse.jetty.server.session.HashSessionManager@4065ec0cTimer, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
21) Thread[id=63, name=qtp236770607-63, state=RUNNABLE, group=TGRP-NoFacetCloudTest]
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)
at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
22) Thread[id=69, name=qtp236770607-69, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
23) Thread[id=51, name=qtp17719963-51, state=RUNNABLE, group=TGRP-NoFacetCloudTest]
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)
at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
24) Thread[id=58, name=qtp17719963-58, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
25) Thread[id=67, name=qtp236770607-67, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
26) Thread[id=53, name=qtp1795747797-53, state=RUNNABLE, group=TGRP-NoFacetCloudTest]
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)
at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
27) Thread[id=84, name=qtp1317280112-84, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
28) Thread[id=56, name=qtp17719963-56, state=RUNNABLE, group=TGRP-NoFacetCloudTest]
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)
at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
29) Thread[id=64, name=qtp236770607-64, state=RUNNABLE, group=TGRP-NoFacetCloudTest]
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)
at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
30) Thread[id=79, name=qtp1795747797-79, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
31) Thread[id=75, name=qtp1317280112-75, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
32) Thread[id=68, name=qtp236770607-68, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
33) Thread[id=78, name=qtp1795747797-78, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
34) Thread[id=59, name=qtp17719963-59, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
35) Thread[id=77, name=qtp1795747797-77, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
36) Thread[id=82, name=org.eclipse.jetty.server.session.HashSessionManager@21fccc9bTimer, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([5ECF0772F96CC9BF]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.analytics.NoFacetCloudTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=52, name=qtp1795747797-52, state=RUNNABLE, group=TGRP-NoFacetCloudTest] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100) at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=72, name=qtp1317280112-72, state=RUNNABLE, group=TGRP-NoFacetCloudTest] at org.slf4j.helpers.MessageFormatter.safeObjectAppend(MessageFormatter.java:305) at org.slf4j.helpers.MessageFormatter.deeplyAppendParameter(MessageFormatter.java:277) at org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:231) at org.eclipse.jetty.util.log.JettyAwareLogger.log(JettyAwareLogger.java:619) at org.eclipse.jetty.util.log.JettyAwareLogger.debug(JettyAwareLogger.java:206) at org.eclipse.jetty.util.log.Slf4jLog.debug(Slf4jLog.java:89) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:245) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100) at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=81, name=org.eclipse.jetty.server.session.HashSessionManager@7dfba6bTimer, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=71, name=qtp1317280112-71, state=RUNNABLE, group=TGRP-NoFacetCloudTest] at sun.nio.ch.EPollArrayWrapper.interrupt(Native Method) at sun.nio.ch.EPollArrayWrapper.interrupt(EPollArrayWrapper.java:317) at sun.nio.ch.EPollSelectorImpl.wakeup(EPollSelectorImpl.java:207) at java.nio.channels.spi.AbstractSelector$1.interrupt(AbstractSelector.java:213) at java.nio.channels.spi.AbstractSelector.begin(AbstractSelector.java:219) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:92) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100) at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=53, name=qtp1795747797-53, state=RUNNABLE, group=TGRP-NoFacetCloudTest] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100) at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589) at java.lang.Thread.run(Thread.java:748) 6) Thread[id=80, name=org.eclipse.jetty.server.session.HashSessionManager@4065ec0cTimer, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 7) Thread[id=86, name=org.eclipse.jetty.server.session.HashSessionManager@2523fec9Timer, state=TIMED_WAITING, group=TGRP-NoFacetCloudTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 8) Thread[id=56, name=qtp17719963-56, state=RUNNABLE, group=TGRP-NoFacetCloudTest] at sun.nio.ch.IOUtil.drain(Native Method) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:104) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecutePro
[...truncated too long message...]
] > at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 6) Thread[id=80, name=qtp1658268881-80, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
[junit4] > at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 7) Thread[id=75, name=qtp1658268881-75-acceptor-0@52d36421-ServerConnector@6fd4abf7{HTTP/1.1,[http/1.1]}{127.0.0.1:51736}, state=RUNNABLE, group=TGRP-TestLTROnSolrCloud]
[junit4] > at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
[junit4] > at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
[junit4] > at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
[junit4] > at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:371)
[junit4] > at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:601)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 8) Thread[id=73, name=qtp1658268881-73, state=RUNNABLE, group=TGRP-TestLTROnSolrCloud]
[junit4] > at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
[junit4] > at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
[junit4] > at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
[junit4] > at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
[junit4] > at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
[junit4] > at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
[junit4] > at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
[junit4] > at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)
[junit4] > at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 9) Thread[id=78, name=qtp1658268881-78, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
[junit4] > at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:563)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([3D4D6423478C7A64]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=74, name=qtp1658268881-74, state=RUNNABLE, group=TGRP-TestLTROnSolrCloud]
[junit4] > at sun.nio.ch.IOUtil.drain(Native Method)
[junit4] > at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:104)
[junit4] > at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
[junit4] > at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
[junit4] > at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
[junit4] > at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
[junit4] > at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)
[junit4] > at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 2) Thread[id=81, name=org.eclipse.jetty.server.session.HashSessionManager@116af2d7Timer, state=TIMED_WAITING, group=TGRP-TestLTROnSolrCloud]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
[junit4] > at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
[junit4] > at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 3) Thread[id=73, name=qtp1658268881-73, state=RUNNABLE, group=TGRP-TestLTROnSolrCloud]
[junit4] > at sun.nio.ch.EPollArrayWrapper.interrupt(Native Method)
[junit4] > at sun.nio.ch.EPollArrayWrapper.interrupt(EPollArrayWrapper.java:317)
[junit4] > at sun.nio.ch.EPollSelectorImpl.wakeup(EPollSelectorImpl.java:207)
[junit4] > at java.nio.channels.spi.AbstractSelector$1.interrupt(AbstractSelector.java:213)
[junit4] > at java.nio.channels.spi.AbstractSelector.begin(AbstractSelector.java:219)
[junit4] > at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:92)
[junit4] > at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
[junit4] > at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
[junit4] > at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
[junit4] > at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:243)
[junit4] > at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:191)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] > at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.execute(ExecuteProduceConsume.java:100)
[junit4] > at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:147)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([3D4D6423478C7A64]:0)
[junit4] Completed [32/33 (25!)] on J0 in 27.34s, 1 test, 3 errors <<< FAILURES!
[...truncated 39187 lines...]
[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 39 - Failure
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/39/
4 tests failed.
FAILED: org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([F9F84486DE86524B]:0)
FAILED: junit.framework.TestSuite.org.apache.lucene.spatial3d.TestGeo3DPoint
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([F9F84486DE86524B]:0)
FAILED: org.apache.solr.update.TestInPlaceUpdatesDistrib.test
Error Message:
inplace_updatable_float didn't match for replica at client: https://127.0.0.1:44657/n/y/collection1 expected:<112.0> but was:<1.0>
Stack Trace:
java.lang.AssertionError: inplace_updatable_float didn't match for replica at client: https://127.0.0.1:44657/n/y/collection1 expected:<112.0> but was:<1.0>
at __randomizedtesting.SeedInfo.seed([AB1BAF000D5237F0:234F90DAA3AE5A08]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.apache.solr.update.TestInPlaceUpdatesDistrib.assertReplicaValue(TestInPlaceUpdatesDistrib.java:980)
at org.apache.solr.update.TestInPlaceUpdatesDistrib.outOfOrderUpdatesIndividualReplicaTest(TestInPlaceUpdatesDistrib.java:643)
at org.apache.solr.update.TestInPlaceUpdatesDistrib.test(TestInPlaceUpdatesDistrib.java:144)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test
Error Message:
Timeout waiting for all live and active
Stack Trace:
java.lang.AssertionError: Timeout waiting for all live and active
at __randomizedtesting.SeedInfo.seed([AB1BAF000D5237F0:234F90DAA3AE5A08]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.testBasics(SharedFSAutoReplicaFailoverTest.java:246)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test(SharedFSAutoReplicaFailoverTest.java:136)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 10455 lines...]
[junit4] Suite: org.apache.lucene.spatial3d.TestGeo3DPoint
[junit4] 2> sep 07, 2017 3:19:58 PM com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
[junit4] 2> WARNING: Suite execution timed out: org.apache.lucene.spatial3d.TestGeo3DPoint
[junit4] 2> 1) Thread[id=1, name=main, state=WAITING, group=main]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Thread.join(Thread.java:1252)
[junit4] 2> at java.lang.Thread.join(Thread.java:1326)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:636)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:493)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:251)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:368)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:13)
[junit4] 2> 2) Thread[id=9, name=JUnit4-serializer-daemon, state=TIMED_WAITING, group=main]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:50)
[junit4] 2> 3) Thread[id=11, name=SUITE-TestGeo3DPoint-seed#[F9F84486DE86524B], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at java.lang.Thread.getStackTrace(Thread.java:1559)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:696)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:693)
[junit4] 2> at java.security.AccessController.doPrivileged(Native Method)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getStackTrace(ThreadLeakControl.java:693)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getThreadsWithTraces(ThreadLeakControl.java:709)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:689)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:65)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:415)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:705)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:139)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:626)
[junit4] 2> 4) Thread[id=12, name=TEST-TestGeo3DPoint.testRandomBig-seed#[F9F84486DE86524B], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at org.apache.lucene.spatial3d.geom.XYZSolidFactory.makeXYZSolid(XYZSolidFactory.java:64)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.verify(TestGeo3DPoint.java:847)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.doTestRandom(TestGeo3DPoint.java:548)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig(TestGeo3DPoint.java:480)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestGeo3DPoint -Dtests.method=testRandomBig -Dtests.seed=F9F84486DE86524B -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=sr-Latn-BA -Dtests.timezone=Australia/LHI -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 6833s J1 | TestGeo3DPoint.testRandomBig <<<
[junit4] > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
[junit4] > at __randomizedtesting.SeedInfo.seed([F9F84486DE86524B]:0)
[junit4] 2> sep 07, 2017 3:19:59 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> sep 07, 2017 3:20:19 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.lucene.spatial3d.TestGeo3DPoint:
[junit4] 2> 1) Thread[id=12, name=TEST-TestGeo3DPoint.testRandomBig-seed#[F9F84486DE86524B], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoComplexPolygon.isWithin(GeoComplexPolygon.java:366)
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoBaseMembershipShape.isWithin(GeoBaseMembershipShape.java:36)
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoBaseShape.getBounds(GeoBaseShape.java:52)
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoComplexPolygon.getBounds(GeoComplexPolygon.java:441)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.verify(TestGeo3DPoint.java:846)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.doTestRandom(TestGeo3DPoint.java:548)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig(TestGeo3DPoint.java:480)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> sep 07, 2017 3:20:19 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=12, name=TEST-TestGeo3DPoint.testRandomBig-seed#[F9F84486DE86524B], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> sep 07, 2017 3:20:22 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> SEVERE: There are still zombie threads that couldn't be terminated:
[junit4] 2> 1) Thread[id=12, name=TEST-TestGeo3DPoint.testRandomBig-seed#[F9F84486DE86524B], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoComplexPolygon.isWithin(GeoComplexPolygon.java:209)
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoBaseMembershipShape.isWithin(GeoBaseMembershipShape.java:36)
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoBaseShape.getBounds(GeoBaseShape.java:52)
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoComplexPolygon.getBounds(GeoComplexPolygon.java:441)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.verify(TestGeo3DPoint.java:846)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.doTestRandom(TestGeo3DPoint.java:548)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig(TestGeo3DPoint.java:480)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {id=FSTOrd50}, docValues:{id=DocValuesFormat(name=Lucene70), point=DocValuesFormat(name=Asserting)}, maxPointsInLeafNode=198, maxMBSortInHeap=6.620287243814608, sim=RandomSimilarity(queryNorm=true): {}, locale=sr-Latn-BA, timezone=Australia/LHI
[junit4] 2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=2,free=145855168,total=412614656
[junit4] 2> NOTE: All tests run in this JVM: [TestGeo3DPoint]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestGeo3DPoint -Dtests.seed=F9F84486DE86524B -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=sr-Latn-BA -Dtests.timezone=Australia/LHI -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J1 | TestGeo3DPoint (suite) <<<
[junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4] > at __randomizedtesting.SeedInfo.seed([F9F84486DE86524B]:0)
[junit4] Completed [13/15 (1!)] on J1 in 7225.18s, 4 tests, 2 errors <<< FAILURES!
[...truncated 2138 lines...]
[junit4] Suite: org.apache.solr.update.TestInPlaceUpdatesDistrib
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.update.TestInPlaceUpdatesDistrib_AB1BAF000D5237F0-001/init-core-data-001
[junit4] 2> 1210048 WARN (SUITE-TestInPlaceUpdatesDistrib-seed#[AB1BAF000D5237F0]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=14 numCloses=14
[junit4] 2> 1210064 INFO (SUITE-TestInPlaceUpdatesDistrib-seed#[AB1BAF000D5237F0]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 1210065 INFO (SUITE-TestInPlaceUpdatesDistrib-seed#[AB1BAF000D5237F0]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 1210066 INFO (SUITE-TestInPlaceUpdatesDistrib-seed#[AB1BAF000D5237F0]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 1210066 INFO (SUITE-TestInPlaceUpdatesDistrib-seed#[AB1BAF000D5237F0]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /n/y
[junit4] 2> 1210067 INFO (SUITE-TestInPlaceUpdatesDistrib-seed#[AB1BAF000D5237F0]-worker) [ ] o.a.s.SolrTestCaseJ4 ####initCore
[junit4] 2> 1210067 INFO (SUITE-TestInPlaceUpdatesDistrib-seed#[AB1BAF000D5237F0]-worker) [ ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/lib, /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
[junit4] 2> 1210303 INFO (SUITE-TestInPlaceUpdatesDistrib-seed#[AB1BAF000D5237F0]-worker) [ ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 1210414 INFO (SUITE-TestInPlaceUpdatesDistrib-seed#[AB1BAF000D5237F0]-worker) [ ] o.a.s.s.IndexSchema [null] Schema name=inplace-updates
[junit4] 2> 1210417 INFO (SUITE-TestInPlaceUpdatesDistrib-seed#[AB1BAF000D5237F0]-worker) [ ] o.a.s.s.IndexSchema Loaded schema inplace-updates/1.6 with uniqueid field id
[junit4] 2> 1210837 INFO (SUITE-TestInPlaceUpdatesDistrib-seed#[AB1BAF000D5237F0]-worker) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@11e079e2
[junit4] 2> 1210936 INFO (SUITE-TestInPlaceUpdatesDistrib-seed#[AB1BAF000D5237F0]-worker) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@11e079e2
[junit4] 2> 1210936 INFO (SUITE-TestInPlaceUpdatesDistrib-seed#[AB1BAF000D5237F0]-worker) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@11e079e2
[junit4] 2> 1211048 INFO (coreLoadExecutor-1615-thread-1) [ x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/lib, /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
[junit4] 2> 1211223 INFO (coreLoadExecutor-1615-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 1211316 INFO (coreLoadExecutor-1615-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=inplace-updates
[junit4] 2> 1211319 INFO (coreLoadExecutor-1615-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema inplace-updates/1.6 with uniqueid field id
[junit4] 2> 1211319 INFO (coreLoadExecutor-1615-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1, trusted=true
[junit4] 2> 1211332 INFO (coreLoadExecutor-1615-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@11e079e2
[junit4] 2> 1211332 INFO (coreLoadExecutor-1615-thread-1) [ x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1211332 INFO (coreLoadExecutor-1615-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.update.TestInPlaceUpdatesDistrib_AB1BAF000D5237F0-001/init-core-data-001/]
[junit4] 2> 1211337 WARN (coreLoadExecutor-1615-thread-1) [ x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1211637 INFO (coreLoadExecutor-1615-thread-1) [ x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1211637 INFO (coreLoadExecutor-1615-thread-1) [ x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1211638 INFO (coreLoadExecutor-1615-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1211638 INFO (coreLoadExecutor-1615-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1211639 INFO (coreLoadExecutor-1615-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1d0062a7[collection1] main]
[junit4] 2> 1211639 WARN (coreLoadExecutor-1615-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage Cannot write to config directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf; switching to use InMemory storage instead.
[junit4] 2> 1211639 INFO (coreLoadExecutor-1615-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1211640 INFO (coreLoadExecutor-1615-thread-1) [ x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577856835866591232
[junit4] 2> 1211902 INFO (SUITE-TestInPlaceUpdatesDistrib-seed#[AB1BAF000D5237F0]-worker) [ ] o.a.s.SolrTestCaseJ4 ####initCore end
[junit4] 2> 1211902 INFO (searcherExecutor-1616-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1d0062a7[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1211933 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1211986 INFO (Thread-2758) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1211986 INFO (Thread-2758) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1212003 ERROR (Thread-2758) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1212086 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.c.ZkTestServer start zk server on port:40045
[junit4] 2> 1212598 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 1212599 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/schema-inplace-updates.xml to /configs/conf1/schema.xml
[junit4] 2> 1212629 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1212630 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 1212631 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 1212631 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 1212661 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 1212662 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 1212663 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1212693 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 1212694 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 1212956 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use TLOG replicas unless explicitly asked otherwise
[junit4] 2> 1213363 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 1213364 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@238c8810{/n/y,null,AVAILABLE}
[junit4] 2> 1213365 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@15c6ea85{SSL,[ssl, http/1.1]}{127.0.0.1:33777}
[junit4] 2> 1213365 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.e.j.s.Server Started @1225833ms
[junit4] 2> 1213365 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.update.TestInPlaceUpdatesDistrib_AB1BAF000D5237F0-001/tempDir-001/control/data, replicaType=NRT, hostContext=/n/y, hostPort=33777, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.update.TestInPlaceUpdatesDistrib_AB1BAF000D5237F0-001/control-001/cores}
[junit4] 2> 1213365 ERROR (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1213395 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
[junit4] 2> 1213395 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1213395 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 1213395 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-09-07T05:19:01.087Z
[junit4] 2> 1213492 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1213492 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.update.TestInPlaceUpdatesDistrib_AB1BAF000D5237F0-001/control-001/solr.xml
[junit4] 2> 1213659 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@11e079e2, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 1213662 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:40045/solr
[junit4] 2> 1213950 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e5ac6ec5f0004, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 1214382 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [n:127.0.0.1:33777_n%2Fy ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1214383 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [n:127.0.0.1:33777_n%2Fy ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:33777_n%2Fy
[junit4] 2> 1214383 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [n:127.0.0.1:33777_n%2Fy ] o.a.s.c.Overseer Overseer (id=98616052264402949-127.0.0.1:33777_n%2Fy-n_0000000000) starting
[junit4] 2> 1214431 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [n:127.0.0.1:33777_n%2Fy ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33777_n%2Fy
[junit4] 2> 1214512 INFO (OverseerStateUpdate-98616052264402949-127.0.0.1:33777_n%2Fy-n_0000000000) [n:127.0.0.1:33777_n%2Fy ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1214820 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [n:127.0.0.1:33777_n%2Fy ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@11e079e2
[junit4] 2> 1214886 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [n:127.0.0.1:33777_n%2Fy ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@11e079e2
[junit4] 2> 1214886 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [n:127.0.0.1:33777_n%2Fy ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@11e079e2
[junit4] 2> 1214888 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [n:127.0.0.1:33777_n%2Fy ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.update.TestInPlaceUpdatesDistrib_AB1BAF000D5237F0-001/control-001/cores
[junit4] 2> 1215255 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1215255 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:40045/solr ready
[junit4] 2> 1215505 INFO (qtp2027226634-5918) [n:127.0.0.1:33777_n%2Fy ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:33777_n%252Fy&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1215583 INFO (OverseerThreadFactory-1626-thread-1-processing-n:127.0.0.1:33777_n%2Fy) [n:127.0.0.1:33777_n%2Fy ] o.a.s.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 1215779 INFO (qtp2027226634-5915) [n:127.0.0.1:33777_n%2Fy ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1215780 INFO (qtp2027226634-5915) [n:127.0.0.1:33777_n%2Fy ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 1215964 INFO (zkCallback-635-thread-1-processing-n:127.0.0.1:33777_n%2Fy) [n:127.0.0.1:33777_n%2Fy ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1216988 INFO (qtp2027226634-5915) [n:127.0.0.1:33777_n%2Fy c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 1217141 INFO (qtp2027226634-5915) [n:127.0.0.1:33777_n%2Fy c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=inplace-updates
[junit4] 2> 1217189 INFO (qtp2027226634-5915) [n:127.0.0.1:33777_n%2Fy c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema inplace-updates/1.6 with uniqueid field id
[junit4] 2> 1217189 INFO (qtp2027226634-5915) [n:127.0.0.1:33777_n%2Fy c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
[junit4] 2> 1217189 INFO (qtp2027226634-5915) [n:127.0.0.1:33777_n%2Fy c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@11e079e2
[junit4] 2> 1217190 INFO (qtp2027226634-5915) [n:127.0.0.1:33777_n%2Fy c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1217204 INFO (qtp2027226634-5915) [n:127.0.0.1:33777_n%2Fy c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.update.TestInPlaceUpdatesDistrib_AB1BAF000D5237F0-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.update.TestInPlaceUpdatesDistrib_AB1BAF000D5237F0-001/control-001/cores/control_collection_shard1_replica_n1/data/]
[junit4] 2> 1217353 WARN (qtp2027226634-5915) [n:127.0.0.1:33777_n%2Fy c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1217580 INFO (qtp2027226634-5915) [n:127.0.0.1:33777_n%2Fy c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1217580 INFO (qtp2027226634-5915) [n:127.0.0.1:33777_n%2Fy c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1217580 INFO (qtp2027226634-5915) [n:127.0.0.1:33777_n%2Fy c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1217581 INFO (qtp2027226634-5915) [n:127.0.0.1:33777_n%2Fy c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1217582 INFO (qtp2027226634-5915) [n:127.0.0.1:33777_n%2Fy c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5f50d736[control_collection_shard1_replica_n1] main]
[junit4] 2> 1217597 INFO (qtp2027226634-5915) [n:127.0.0.1:33777_n%2Fy c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1217601 INFO (qtp2027226634-5915) [n:127.0.0.1:33777_n%2Fy c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1217602 INFO (qtp2027226634-5915) [n:127.0.0.1:33777_n%2Fy c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1217640 INFO (searcherExecutor-1629-thread-1-processing-n:127.0.0.1:33777_n%2Fy x:control_collection_shard1_replica_n1 s:shard1 c:control_collection) [n:127.0.0.1:33777_n%2Fy c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@5f50d736[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1217640 INFO (qtp2027226634-5915) [n:127.0.0.1:33777_n%2Fy c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577856842158047232
[junit4] 2> 1217679 INFO (qtp2027226634-5915) [n:127.0.0.1:33777_n%2Fy c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1217679 INFO (qtp2027226634-5915) [n:127.0.0.1:33777_n%2Fy c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1217679 INFO (qtp2027226634-5915) [n:127.0.0.1:33777_n%2Fy c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:33777/n/y/control_collection_shard1_replica_n1/
[junit4] 2> 1217679 INFO (qtp2027226634-5915) [n:127.0.0.1:33777_n%2Fy c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1217679 INFO (qtp2027226634-5915) [n:127.0.0.1:33777_n%2Fy c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy https://127.0.0.1:33777/n/y/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 1217679 INFO (qtp2027226634-5915) [n:127.0.0.1:33777_n%2Fy c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 1217710 INFO (qtp2027226634-5915) [n:127.0.0.1:33777_n%2Fy c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:33777/n/y/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 1217822 INFO (zkCallback-635-thread-1-processing-n:127.0.0.1:33777_n%2Fy) [n:127.0.0.1:33777_n%2Fy ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1217872 INFO (qtp2027226634-5915) [n:127.0.0.1:33777_n%2Fy c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1217873 INFO (qtp2027226634-5915) [n:127.0.0.1:33777_n%2Fy c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2093
[junit4] 2> 1217905 INFO (qtp2027226634-5918) [n:127.0.0.1:33777_n%2Fy ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 1218022 INFO (zkCallback-635-thread-1-processing-n:127.0.0.1:33777_n%2Fy) [n:127.0.0.1:33777_n%2Fy ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1218907 INFO (qtp2027226634-5918) [n:127.0.0.1:33777_n%2Fy ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:33777_n%252Fy&wt=javabin&version=2} status=0 QTime=3402
[junit4] 2> 1218974 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1218974 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:40045/solr ready
[junit4] 2> 1218974 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 1218992 INFO (qtp2027226634-5911) [n:127.0.0.1:33777_n%2Fy ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&stateFormat=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1219054 INFO (OverseerThreadFactory-1626-thread-2-processing-n:127.0.0.1:33777_n%2Fy) [n:127.0.0.1:33777_n%2Fy ] o.a.s.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 1219054 INFO (OverseerCollectionConfigSetProcessor-98616052264402949-127.0.0.1:33777_n%2Fy-n_0000000000) [n:127.0.0.1:33777_n%2Fy ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1219055 WARN (OverseerThreadFactory-1626-thread-2-processing-n:127.0.0.1:33777_n%2Fy) [n:127.0.0.1:33777_n%2Fy ] o.a.s.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
[junit4] 2> 1219270 INFO (qtp2027226634-5911) [n:127.0.0.1:33777_n%2Fy ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 1219270 INFO (qtp2027226634-5911) [n:127.0.0.1:33777_n%2Fy ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&stateFormat=1&wt=javabin&version=2} status=0 QTime=277
[junit4] 2> 1226265 WARN (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]-SendThread(127.0.0.1:40045)) [ ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 7091ms for sessionid 0x15e5ac6ec5f0007
[junit4] 2> 1226297 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e5ac6ec5f0007, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 1226322 INFO (OverseerCollectionConfigSetProcessor-98616052264402949-127.0.0.1:33777_n%2Fy-n_0000000000) [n:127.0.0.1:33777_n%2Fy ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1226521 WARN (zkCallback-642-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6dd714fd name: ZooKeeperConnection Watcher:127.0.0.1:40045/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
[junit4] 2> 1226521 WARN (zkCallback-642-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has disconnected
[junit4] 2> 1227081 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.update.TestInPlaceUpdatesDistrib_AB1BAF000D5237F0-001/shard-1-001 of type TLOG
[junit4] 2> 1227111 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 1227240 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@245a551c{/n/y,null,AVAILABLE}
[junit4] 2> 1227241 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3154c910{SSL,[ssl, http/1.1]}{127.0.0.1:42099}
[junit4] 2> 1227241 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.e.j.s.Server Started @1239709ms
[junit4] 2> 1227241 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.update.TestInPlaceUpdatesDistrib_AB1BAF000D5237F0-001/tempDir-001/jetty1, replicaType=TLOG, solrconfig=solrconfig.xml, hostContext=/n/y, hostPort=42099, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.update.TestInPlaceUpdatesDistrib_AB1BAF000D5237F0-001/shard-1-001/cores}
[junit4] 2> 1227241 ERROR (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1227241 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
[junit4] 2> 1227241 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1227241 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 1227241 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-09-07T05:19:14.933Z
[junit4] 2> 1227427 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1227427 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.update.TestInPlaceUpdatesDistrib_AB1BAF000D5237F0-001/shard-1-001/solr.xml
[junit4] 2> 1227444 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@11e079e2, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 1227476 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:40045/solr
[junit4] 2> 1227629 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [n:127.0.0.1:42099_n%2Fy ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1227660 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [n:127.0.0.1:42099_n%2Fy ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1227674 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [n:127.0.0.1:42099_n%2Fy ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42099_n%2Fy
[junit4] 2> 1227699 INFO (zkCallback-635-thread-1-processing-n:127.0.0.1:33777_n%2Fy) [n:127.0.0.1:33777_n%2Fy ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1227821 INFO (zkCallback-642-thread-5) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1227939 INFO (zkCallback-647-thread-1-processing-n:127.0.0.1:42099_n%2Fy) [n:127.0.0.1:42099_n%2Fy ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1228335 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [n:127.0.0.1:42099_n%2Fy ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@11e079e2
[junit4] 2> 1228368 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [n:127.0.0.1:42099_n%2Fy ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@11e079e2
[junit4] 2> 1228369 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [n:127.0.0.1:42099_n%2Fy ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@11e079e2
[junit4] 2> 1228382 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [n:127.0.0.1:42099_n%2Fy ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.update.TestInPlaceUpdatesDistrib_AB1BAF000D5237F0-001/shard-1-001/cores
[junit4] 2> 1228508 INFO (qtp2027226634-5914) [n:127.0.0.1:33777_n%2Fy ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:42099_n%252Fy&action=ADDREPLICA&collection=collection1&shard=shard1&type=TLOG&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1228510 INFO (OverseerThreadFactory-1626-thread-3-processing-n:127.0.0.1:33777_n%2Fy) [n:127.0.0.1:33777_n%2Fy ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:42099_n%2Fy for creating new replica
[junit4] 2> 1228755 INFO (qtp1979647043-5958) [n:127.0.0.1:42099_n%2Fy ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_t21&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=TLOG
[junit4] 2> 1228775 INFO (qtp1979647043-5958) [n:127.0.0.1:42099_n%2Fy ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 1228959 INFO (qtp1979647043-5958) [n:127.0.0.1:42099_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t21] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 1229060 INFO (qtp1979647043-5958) [n:127.0.0.1:42099_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t21] o.a.s.s.IndexSchema [collection1_shard1_replica_t21] Schema name=inplace-updates
[junit4] 2> 1229063 INFO (qtp1979647043-5958) [n:127.0.0.1:42099_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t21] o.a.s.s.IndexSchema Loaded schema inplace-updates/1.6 with uniqueid field id
[junit4] 2> 1229063 INFO (qtp1979647043-5958) [n:127.0.0.1:42099_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t21] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_t21' using configuration from collection collection1, trusted=true
[junit4] 2> 1229064 INFO (qtp1979647043-5958) [n:127.0.0.1:42099_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t21] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_t21' (registry 'solr.core.collection1.shard1.replica_t21') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@11e079e2
[junit4] 2> 1229092 INFO (qtp1979647043-5958) [n:127.0.0.1:42099_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t21] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1229092 INFO (qtp1979647043-5958) [n:127.0.0.1:42099_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t21] o.a.s.c.SolrCore [[collection1_shard1_replica_t21] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.update.TestInPlaceUpdatesDistrib_AB1BAF000D5237F0-001/shard-1-001/cores/collection1_shard1_replica_t21], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.update.TestInPlaceUpdatesDistrib_AB1BAF000D5237F0-001/shard-1-001/cores/collection1_shard1_replica_t21/data/]
[junit4] 2> 1229118 WARN (qtp1979647043-5958) [n:127.0.0.1:42099_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t21] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1229576 INFO (qtp1979647043-5958) [n:127.0.0.1:42099_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t21] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1229576 INFO (qtp1979647043-5958) [n:127.0.0.1:42099_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t21] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1229578 INFO (qtp1979647043-5958) [n:127.0.0.1:42099_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t21] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1229578 INFO (qtp1979647043-5958) [n:127.0.0.1:42099_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t21] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1229579 INFO (qtp1979647043-5958) [n:127.0.0.1:42099_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t21] o.a.s.s.SolrIndexSearcher Opening [Searcher@9a421ae[collection1_shard1_replica_t21] main]
[junit4] 2> 1229610 INFO (qtp1979647043-5958) [n:127.0.0.1:42099_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t21] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1229610 INFO (qtp1979647043-5958) [n:127.0.0.1:42099_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t21] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1229611 INFO (qtp1979647043-5958) [n:127.0.0.1:42099_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t21] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1229611 INFO (qtp1979647043-5958) [n:127.0.0.1:42099_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t21] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577856854710550528
[junit4] 2> 1229691 INFO (searcherExecutor-1640-thread-1-processing-n:127.0.0.1:42099_n%2Fy x:collection1_shard1_replica_t21 s:shard1 c:collection1) [n:127.0.0.1:42099_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t21] o.a.s.c.SolrCore [collection1_shard1_replica_t21] Registered new searcher Searcher@9a421ae[collection1_shard1_replica_t21] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1229722 INFO (qtp1979647043-5958) [n:127.0.0.1:42099_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t21] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1229722 INFO (qtp1979647043-5958) [n:127.0.0.1:42099_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t21] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1229722 INFO (qtp1979647043-5958) [n:127.0.0.1:42099_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t21] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:42099/n/y/collection1_shard1_replica_t21/
[junit4] 2> 1229722 INFO (qtp1979647043-5958) [n:127.0.0.1:42099_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t21] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1229722 INFO (qtp1979647043-5958) [n:127.0.0.1:42099_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t21] o.a.s.c.SyncStrategy https://127.0.0.1:42099/n/y/collection1_shard1_replica_t21/ has no replicas
[junit4] 2> 1229722 INFO (qtp1979647043-5958) [n:127.0.0.1:42099_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t21] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 1229722 INFO (qtp1979647043-5958) [n:127.0.0.1:42099_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t21] o.a.s.c.ZkController collection1_shard1_replica_t21 stopping background replication from leader
[junit4] 2> 1229724 INFO (qtp1979647043-5958) [n:127.0.0.1:42099_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t21] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:42099/n/y/collection1_shard1_replica_t21/ shard1
[junit4] 2> 1229937 INFO (qtp1979647043-5958) [n:127.0.0.1:42099_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t21] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1229938 INFO (qtp1979647043-5958) [n:127.0.0.1:42099_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t21] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_t21&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=1183
[junit4] 2> 1229978 INFO (qtp2027226634-5914) [n:127.0.0.1:33777_n%2Fy ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:42099_n%252Fy&action=ADDREPLICA&collection=collection1&shard=shard1&type=TLOG&wt=javabin&version=2} status=0 QTime=1470
[junit4] 2> 1230519 INFO (OverseerCollectionConfigSetProcessor-98616052264402949-127.0.0.1:33777_n%2Fy-n_0000000000) [n:127.0.0.1:33777_n%2Fy ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1230958 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.update.TestInPlaceUpdatesDistrib_AB1BAF000D5237F0-001/shard-2-001 of type TLOG
[junit4] 2> 1230990 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 1231120 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@552ceed2{/n/y,null,AVAILABLE}
[junit4] 2> 1231121 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3934b43b{SSL,[ssl, http/1.1]}{127.0.0.1:44657}
[junit4] 2> 1231121 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.e.j.s.Server Started @1243590ms
[junit4] 2> 1231121 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.update.TestInPlaceUpdatesDistrib_AB1BAF000D5237F0-001/tempDir-001/jetty2, replicaType=TLOG, solrconfig=solrconfig.xml, hostContext=/n/y, hostPort=44657, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.update.TestInPlaceUpdatesDistrib_AB1BAF000D5237F0-001/shard-2-001/cores}
[junit4] 2> 1231152 ERROR (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1231153 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
[junit4] 2> 1231153 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1231153 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 1231153 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-09-07T05:19:18.845Z
[junit4] 2> 1231252 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1231252 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.update.TestInPlaceUpdatesDistrib_AB1BAF000D5237F0-001/shard-2-001/solr.xml
[junit4] 2> 1231354 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@11e079e2, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 1231433 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:40045/solr
[junit4] 2> 1231748 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [n:127.0.0.1:44657_n%2Fy ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1231786 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [n:127.0.0.1:44657_n%2Fy ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1231787 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [n:127.0.0.1:44657_n%2Fy ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44657_n%2Fy
[junit4] 2> 1231788 INFO (zkCallback-647-thread-1-processing-n:127.0.0.1:42099_n%2Fy) [n:127.0.0.1:42099_n%2Fy ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1231789 INFO (zkCallback-653-thread-1-processing-n:127.0.0.1:44657_n%2Fy) [n:127.0.0.1:44657_n%2Fy ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1231789 INFO (zkCallback-642-thread-5) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1231808 INFO (zkCallback-635-thread-1-processing-n:127.0.0.1:33777_n%2Fy) [n:127.0.0.1:33777_n%2Fy ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1232373 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [n:127.0.0.1:44657_n%2Fy ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@11e079e2
[junit4] 2> 1232461 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [n:127.0.0.1:44657_n%2Fy ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@11e079e2
[junit4] 2> 1232461 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [n:127.0.0.1:44657_n%2Fy ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@11e079e2
[junit4] 2> 1232462 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[AB1BAF000D5237F0]) [n:127.0.0.1:44657_n%2Fy ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.update.TestInPlaceUpdatesDistrib_AB1BAF000D5237F0-001/shard-2-001/cores
[junit4] 2> 1232936 INFO (qtp2027226634-5915) [n:127.0.0.1:33777_n%2Fy ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:44657_n%252Fy&action=ADDREPLICA&collection=collection1&shard=shard1&type=TLOG&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1232973 INFO (OverseerThreadFactory-1626-thread-4-processing-n:127.0.0.1:33777_n%2Fy) [n:127.0.0.1:33777_n%2Fy ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:44657_n%2Fy for creating new replica
[junit4] 2> 1233298 INFO (qtp1394263838-5993) [n:127.0.0.1:44657_n%2Fy ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_t23&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=TLOG
[junit4] 2> 1233298 INFO (qtp1394263838-5993) [n:127.0.0.1:44657_n%2Fy ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 1233412 INFO (qtp1394263838-5993) [n:127.0.0.1:44657_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t23] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 1233450 INFO (qtp1394263838-5993) [n:127.0.0.1:44657_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t23] o.a.s.s.IndexSchema [collection1_shard1_replica_t23] Schema name=inplace-updates
[junit4] 2> 1233481 INFO (qtp1394263838-5993) [n:127.0.0.1:44657_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t23] o.a.s.s.IndexSchema Loaded schema inplace-updates/1.6 with uniqueid field id
[junit4] 2> 1233481 INFO (qtp1394263838-5993) [n:127.0.0.1:44657_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t23] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_t23' using configuration from collection collection1, trusted=true
[junit4] 2> 1233481 INFO (qtp1394263838-5993) [n:127.0.0.1:44657_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t23] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_t23' (registry 'solr.core.collection1.shard1.replica_t23') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@11e079e2
[junit4] 2> 1233481 INFO (qtp1394263838-5993) [n:127.0.0.1:44657_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t23] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1233481 INFO (qtp1394263838-5993) [n:127.0.0.1:44657_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t23] o.a.s.c.SolrCore [[collection1_shard1_replica_t23] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.update.TestInPlaceUpdatesDistrib_AB1BAF000D5237F0-001/shard-2-001/cores/collection1_shard1_replica_t23], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.update.TestInPlaceUpdatesDistrib_AB1BAF000D5237F0-001/shard-2-001/cores/collection1_shard1_replica_t23/data/]
[junit4] 2> 1233529 WARN (qtp1394263838-5993) [n:127.0.0.1:44657_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t23] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1233677 INFO (qtp1394263838-5993) [n:127.0.0.1:44657_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t23] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1233677 INFO (qtp1394263838-5993) [n:127.0.0.1:44657_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t23] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1233678 INFO (qtp1394263838-5993) [n:127.0.0.1:44657_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t23] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1233678 INFO (qtp1394263838-5993) [n:127.0.0.1:44657_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t23] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1233692 INFO (qtp1394263838-5993) [n:127.0.0.1:44657_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t23] o.a.s.s.SolrIndexSearcher Opening [Searcher@4539ef9c[collection1_shard1_replica_t23] main]
[junit4] 2> 1233693 INFO (qtp1394263838-5993) [n:127.0.0.1:44657_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t23] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1233693 INFO (qtp1394263838-5993) [n:127.0.0.1:44657_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t23] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1233693 INFO (qtp1394263838-5993) [n:127.0.0.1:44657_n%2Fy c:collection1 s:shard1 x:collection1_shard1_replica_t23] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1233695 INFO (searcherExecutor-1651-thread-1-processing-n:127.0.0.1:44657_n%2Fy x:collectio
[...truncated too long message...]
[junit4] 2> 1811569 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[AB1BAF000D5237F0]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:37774 37774
[junit4] 2> 1811762 INFO (Thread-4149) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:37774 37774
[junit4] 2> 1811796 WARN (Thread-4149) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 13 /solr/aliases.json
[junit4] 2> 10 /solr/security.json
[junit4] 2> 10 /solr/configs/conf1
[junit4] 2> 4 /solr/collections/collection1/state.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 126 /solr/collections/solrj_collection4/state.json
[junit4] 2> 118 /solr/collections/solrj_collection2/state.json
[junit4] 2> 115 /solr/collections/solrj_collection3/state.json
[junit4] 2> 112 /solr/collections/solrj_collection/state.json
[junit4] 2> 13 /solr/clusterprops.json
[junit4] 2> 13 /solr/clusterstate.json
[junit4] 2> 3 /solr/overseer_elect/election/98616073279766537-127.0.0.1:39258__s-n_0000000001
[junit4] 2> 2 /solr/overseer_elect/election/98616073279766532-127.0.0.1:45607__s-n_0000000000
[junit4] 2> 2 /solr/collections/solrj_collection/leader_elect/shard2/election/98616073279766546-core_node8-n_0000000000
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 13 /solr/collections
[junit4] 2> 8 /solr/live_nodes
[junit4] 2> 5 /solr/overseer/queue
[junit4] 2> 5 /solr/overseer/collection-queue-work
[junit4] 2> 4 /solr/overseer/queue-work
[junit4] 2>
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=SharedFSAutoReplicaFailoverTest -Dtests.method=test -Dtests.seed=AB1BAF000D5237F0 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=el -Dtests.timezone=Israel -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] FAILURE 281s J2 | SharedFSAutoReplicaFailoverTest.test <<<
[junit4] > Throwable #1: java.lang.AssertionError: Timeout waiting for all live and active
[junit4] > at __randomizedtesting.SeedInfo.seed([AB1BAF000D5237F0:234F90DAA3AE5A08]:0)
[junit4] > at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.testBasics(SharedFSAutoReplicaFailoverTest.java:246)
[junit4] > at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test(SharedFSAutoReplicaFailoverTest.java:136)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 1811799 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[AB1BAF000D5237F0]-worker) [ ] o.a.h.h.s.d.DirectoryScanner DirectoryScanner: shutdown has been called
[junit4] 2> 1811913 WARN (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_AB1BAF000D5237F0-001/tempDir-001/hdfsBaseDir/data/data3/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_AB1BAF000D5237F0-001/tempDir-001/hdfsBaseDir/data/data4/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:35463) [ ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-1499023873-127.0.0.1-1504761853485 (Datanode Uuid 17cb0cb6-7763-4abc-9333-10a9284e4b3d) service to lucene2-us-west.apache.org/127.0.0.1:35463
[junit4] 2> 1812288 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[AB1BAF000D5237F0]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
[junit4] 2> 1812388 WARN (1705596261@qtp-1189476991-1 - Acceptor0 HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42917) [ ] o.a.h.h.HttpServer2 HttpServer Acceptor: isRunning is false. Rechecking.
[junit4] 2> 1812388 WARN (1705596261@qtp-1189476991-1 - Acceptor0 HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42917) [ ] o.a.h.h.HttpServer2 HttpServer Acceptor: isRunning is false
[junit4] 2> 1812486 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[AB1BAF000D5237F0]-worker) [ ] o.a.h.h.s.d.DirectoryScanner DirectoryScanner: shutdown has been called
[junit4] 2> 1812889 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[AB1BAF000D5237F0]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
[junit4] 2> 1812912 WARN (223217811@qtp-332094130-1 - Acceptor0 HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41358) [ ] o.a.h.h.HttpServer2 HttpServer Acceptor: isRunning is false. Rechecking.
[junit4] 2> 1812912 WARN (223217811@qtp-332094130-1 - Acceptor0 HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:41358) [ ] o.a.h.h.HttpServer2 HttpServer Acceptor: isRunning is false
[junit4] 2> 1812990 WARN (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_AB1BAF000D5237F0-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_AB1BAF000D5237F0-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:35463) [ ] o.a.h.h.s.d.IncrementalBlockReportManager IncrementalBlockReportManager interrupted
[junit4] 2> 1812990 WARN (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_AB1BAF000D5237F0-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_AB1BAF000D5237F0-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:35463) [ ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-1499023873-127.0.0.1-1504761853485 (Datanode Uuid 8b5abeb5-45ca-40ab-b2a1-09543f9cc8f8) service to lucene2-us-west.apache.org/127.0.0.1:35463
[junit4] 2> 1813234 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[AB1BAF000D5237F0]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@lucene2-us-west.apache.org:0
[junit4] 2> 1813248 WARN (280695654@qtp-444004908-1 - Acceptor0 HttpServer2$SelectChannelConnectorWithSafeStartup@lucene2-us-west.apache.org:39921) [ ] o.a.h.h.HttpServer2 HttpServer Acceptor: isRunning is false. Rechecking.
[junit4] 2> 1813249 WARN (280695654@qtp-444004908-1 - Acceptor0 HttpServer2$SelectChannelConnectorWithSafeStartup@lucene2-us-west.apache.org:39921) [ ] o.a.h.h.HttpServer2 HttpServer Acceptor: isRunning is false
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_AB1BAF000D5237F0-001
[junit4] 2> Sep 07, 2017 5:29:03 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 34 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {id=PostingsFormat(name=Memory), text=PostingsFormat(name=LuceneVarGapDocFreqInterval)}, docValues:{_version_=DocValuesFormat(name=Direct)}, maxPointsInLeafNode=1575, maxMBSortInHeap=5.301961939838006, sim=RandomSimilarity(queryNorm=true): {}, locale=el, timezone=Israel
[junit4] 2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=2,free=203541104,total=512753664
[junit4] 2> NOTE: All tests run in this JVM: [TestZkChroot, TestPolicyCloud, TestRecovery, MinimalSchemaTest, TestManagedResourceStorage, TestSuggestSpellingConverter, SegmentsInfoRequestHandlerTest, TestNamedUpdateProcessors, SolrShardReporterTest, BlockJoinFacetSimpleTest, TestUseDocValuesAsStored2, DirectUpdateHandlerOptimizeTest, CollectionTooManyReplicasTest, TestDistributedStatsComponentCardinality, HdfsDirectoryTest, TestSolrCloudWithSecureImpersonation, TestLRUStatsCache, DocValuesMissingTest, SpellPossibilityIteratorTest, SolrTestCaseJ4Test, JvmMetricsTest, SearchHandlerTest, TestBulkSchemaConcurrent, DateFieldTest, SharedFSAutoReplicaFailoverTest]
[junit4] Completed [146/731 (2!)] on J2 in 298.47s, 1 test, 1 failure <<< FAILURES!
[...truncated 48251 lines...]
BUILD FAILED
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/build.xml:818: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/build.xml:690: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/build.xml:678: Source checkout is modified!!! Offending files:
* solr/licenses/commons-math3-3.6.1.jar.sha1
Total time: 756 minutes 36 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 38 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/38/
7 tests failed.
FAILED: org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest.test
Error Message:
The Monkey ran for over 45 seconds and no jetties were stopped - this is worth investigating!
Stack Trace:
java.lang.AssertionError: The Monkey ran for over 45 seconds and no jetties were stopped - this is worth investigating!
at __randomizedtesting.SeedInfo.seed([A85C3D2AC3D0CB06:200802F06D2CA6FE]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.ChaosMonkey.stopTheMonkey(ChaosMonkey.java:587)
at org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest.test(ChaosMonkeySafeLeaderWithPullReplicasTest.java:174)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest
Error Message:
9 threads leaked from SUITE scope at org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest: 1) Thread[id=152206, name=StoppableCommitThread-EventThread, state=WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501) 2) Thread[id=152100, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at java.lang.Thread.sleep(Native Method) at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135) 3) Thread[id=152207, name=zkCallback-26492-thread-1, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=152277, name=zkCallback-26492-thread-3, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=152205, name=StoppableCommitThread-SendThread(127.0.0.1:37221), state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060) 6) Thread[id=152278, name=zkCallback-26492-thread-4, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 7) Thread[id=152101, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at java.lang.Thread.sleep(Native Method) at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135) 8) Thread[id=152102, name=StoppableCommitThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at java.lang.Thread.sleep(Native Method) at org.apache.solr.cloud.StoppableCommitThread.run(StoppableCommitThread.java:55) 9) Thread[id=152276, name=zkCallback-26492-thread-2, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 9 threads leaked from SUITE scope at org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest:
1) Thread[id=152206, name=StoppableCommitThread-EventThread, state=WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)
2) Thread[id=152100, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135)
3) Thread[id=152207, name=zkCallback-26492-thread-1, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
4) Thread[id=152277, name=zkCallback-26492-thread-3, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
5) Thread[id=152205, name=StoppableCommitThread-SendThread(127.0.0.1:37221), state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)
6) Thread[id=152278, name=zkCallback-26492-thread-4, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
7) Thread[id=152101, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135)
8) Thread[id=152102, name=StoppableCommitThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.solr.cloud.StoppableCommitThread.run(StoppableCommitThread.java:55)
9) Thread[id=152276, name=zkCallback-26492-thread-2, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([A85C3D2AC3D0CB06]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=152100, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at org.apache.solr.common.util.JavaBinCodec.writeStr(JavaBinCodec.java:841) at org.apache.solr.common.util.JavaBinCodec.writePrimitive(JavaBinCodec.java:932) at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:328) at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228) at org.apache.solr.common.util.JavaBinCodec.writeSolrInputDocument(JavaBinCodec.java:616) at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:355) at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228) at org.apache.solr.common.util.JavaBinCodec.writeMapEntry(JavaBinCodec.java:764) at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:383) at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228) at org.apache.solr.common.util.JavaBinCodec.writeIterator(JavaBinCodec.java:705) at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:367) at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228) at org.apache.solr.common.util.JavaBinCodec.writeNamedList(JavaBinCodec.java:223) at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:330) at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228) at org.apache.solr.common.util.JavaBinCodec.marshal(JavaBinCodec.java:155) at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.marshal(JavaBinUpdateRequestCodec.java:91) at org.apache.solr.client.solrj.impl.BinaryRequestWriter.getContentStream(BinaryRequestWriter.java:67) at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:94) at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:104) at org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:382) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195) at org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:176) at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:115) 2) Thread[id=152207, name=zkCallback-26492-thread-1, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=152277, name=zkCallback-26492-thread-3, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=152205, name=StoppableCommitThread-SendThread(127.0.0.1:37221), state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060) 5) Thread[id=152278, name=zkCallback-26492-thread-4, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 6) Thread[id=152101, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at java.lang.Throwable.getStackTraceElement(Native Method) at java.lang.Throwable.getOurStackTrace(Throwable.java:827) at java.lang.Throwable.printStackTrace(Throwable.java:656) at java.lang.Throwable.printStackTrace(Throwable.java:643) at java.lang.Throwable.printStackTrace(Throwable.java:634) at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:121) 7) Thread[id=152276, name=zkCallback-26492-thread-2, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=152100, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at org.apache.solr.common.util.JavaBinCodec.writeStr(JavaBinCodec.java:841)
at org.apache.solr.common.util.JavaBinCodec.writePrimitive(JavaBinCodec.java:932)
at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:328)
at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
at org.apache.solr.common.util.JavaBinCodec.writeSolrInputDocument(JavaBinCodec.java:616)
at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:355)
at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
at org.apache.solr.common.util.JavaBinCodec.writeMapEntry(JavaBinCodec.java:764)
at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:383)
at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
at org.apache.solr.common.util.JavaBinCodec.writeIterator(JavaBinCodec.java:705)
at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:367)
at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
at org.apache.solr.common.util.JavaBinCodec.writeNamedList(JavaBinCodec.java:223)
at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:330)
at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
at org.apache.solr.common.util.JavaBinCodec.marshal(JavaBinCodec.java:155)
at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.marshal(JavaBinUpdateRequestCodec.java:91)
at org.apache.solr.client.solrj.impl.BinaryRequestWriter.getContentStream(BinaryRequestWriter.java:67)
at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:94)
at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:104)
at org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:382)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195)
at org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:176)
at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:115)
2) Thread[id=152207, name=zkCallback-26492-thread-1, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=152277, name=zkCallback-26492-thread-3, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
4) Thread[id=152205, name=StoppableCommitThread-SendThread(127.0.0.1:37221), state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)
5) Thread[id=152278, name=zkCallback-26492-thread-4, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
6) Thread[id=152101, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at java.lang.Throwable.getStackTraceElement(Native Method)
at java.lang.Throwable.getOurStackTrace(Throwable.java:827)
at java.lang.Throwable.printStackTrace(Throwable.java:656)
at java.lang.Throwable.printStackTrace(Throwable.java:643)
at java.lang.Throwable.printStackTrace(Throwable.java:634)
at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:121)
7) Thread[id=152276, name=zkCallback-26492-thread-2, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([A85C3D2AC3D0CB06]:0)
FAILED: org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_dv
Error Message:
java.lang.RuntimeException: Error from server at http://127.0.0.1:35023/solr/test_col: Async exception during distributed update: Error from server at http://127.0.0.1:39303/solr/test_col_shard1_replica_n1: Server Error request: http://127.0.0.1:39303/solr/test_col_shard1_replica_n1/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A35023%2Fsolr%2Ftest_col_shard4_replica_n14%2F&wt=javabin&version=2 Remote error message: Failed synchronous update on shard StdNode: http://127.0.0.1:43415/solr/test_col_shard1_replica_n2/ update: org.apache.solr.client.solrj.request.UpdateRequest@5dcdad3a
Stack Trace:
java.util.concurrent.ExecutionException: java.lang.RuntimeException: Error from server at http://127.0.0.1:35023/solr/test_col: Async exception during distributed update: Error from server at http://127.0.0.1:39303/solr/test_col_shard1_replica_n1: Server Error
request: http://127.0.0.1:39303/solr/test_col_shard1_replica_n1/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A35023%2Fsolr%2Ftest_col_shard4_replica_n14%2F&wt=javabin&version=2
Remote error message: Failed synchronous update on shard StdNode: http://127.0.0.1:43415/solr/test_col_shard1_replica_n2/ update: org.apache.solr.client.solrj.request.UpdateRequest@5dcdad3a
at __randomizedtesting.SeedInfo.seed([A85C3D2AC3D0CB06:9E485F6C498DF117]:0)
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.checkField(TestStressCloudBlindAtomicUpdates.java:283)
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_dv(TestStressCloudBlindAtomicUpdates.java:195)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: Error from server at http://127.0.0.1:35023/solr/test_col: Async exception during distributed update: Error from server at http://127.0.0.1:39303/solr/test_col_shard1_replica_n1: Server Error
request: http://127.0.0.1:39303/solr/test_col_shard1_replica_n1/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A35023%2Fsolr%2Ftest_col_shard4_replica_n14%2F&wt=javabin&version=2
Remote error message: Failed synchronous update on shard StdNode: http://127.0.0.1:43415/solr/test_col_shard1_replica_n2/ update: org.apache.solr.client.solrj.request.UpdateRequest@5dcdad3a
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.run(TestStressCloudBlindAtomicUpdates.java:411)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:35023/solr/test_col: Async exception during distributed update: Error from server at http://127.0.0.1:39303/solr/test_col_shard1_replica_n1: Server Error
request: http://127.0.0.1:39303/solr/test_col_shard1_replica_n1/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F127.0.0.1%3A35023%2Fsolr%2Ftest_col_shard4_replica_n14%2F&wt=javabin&version=2
Remote error message: Failed synchronous update on shard StdNode: http://127.0.0.1:43415/solr/test_col_shard1_replica_n2/ update: org.apache.solr.client.solrj.request.UpdateRequest@5dcdad3a
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:627)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195)
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.doRandomAtomicUpdate(TestStressCloudBlindAtomicUpdates.java:370)
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.run(TestStressCloudBlindAtomicUpdates.java:399)
... 6 more
FAILED: org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload
Error Message:
expected:<[{indexVersion=1504666109069,generation=2,filelist=[_e35.fdt, _e35.fdx, _e35.fnm, _e35.nvd, _e35.nvm, _e35.si, _e35_Asserting_0.doc, _e35_Asserting_0.tim, _e35_Asserting_0.tip, _e8g.fdt, _e8g.fdx, _e8g.fnm, _e8g.nvd, _e8g.nvm, _e8g.si, _e8g_Asserting_0.doc, _e8g_Asserting_0.tim, _e8g_Asserting_0.tip, _e8h.fdt, _e8h.fdx, _e8h.fnm, _e8h.nvd, _e8h.nvm, _e8h.si, _e8h_Asserting_0.doc, _e8h_Asserting_0.tim, _e8h_Asserting_0.tip, _e8i.fdt, _e8i.fdx, _e8i.fnm, _e8i.nvd, _e8i.nvm, _e8i.si, _e8i_Asserting_0.doc, _e8i_Asserting_0.tim, _e8i_Asserting_0.tip, _e8j.fdt, _e8j.fdx, _e8j.fnm, _e8j.nvd, _e8j.nvm, _e8j.si, _e8j_Asserting_0.doc, _e8j_Asserting_0.tim, _e8j_Asserting_0.tip, _e8k.fdt, _e8k.fdx, _e8k.fnm, _e8k.nvd, _e8k.nvm, _e8k.si, _e8k_Asserting_0.doc, _e8k_Asserting_0.tim, _e8k_Asserting_0.tip, _e8l.fdt, _e8l.fdx, _e8l.fnm, _e8l.nvd, _e8l.nvm, _e8l.si, _e8l_Asserting_0.doc, _e8l_Asserting_0.tim, _e8l_Asserting_0.tip, _e8m.fdt, _e8m.fdx, _e8m.fnm, _e8m.nvd, _e8m.nvm, _e8m.si, _e8m_Asserting_0.doc, _e8m_Asserting_0.tim, _e8m_Asserting_0.tip, _e8n.fdt, _e8n.fdx, _e8n.fnm, _e8n.nvd, _e8n.nvm, _e8n.si, _e8n_Asserting_0.doc, _e8n_Asserting_0.tim, _e8n_Asserting_0.tip, _e8o.fdt, _e8o.fdx, _e8o.fnm, _e8o.nvd, _e8o.nvm, _e8o.si, _e8o_Asserting_0.doc, _e8o_Asserting_0.tim, _e8o_Asserting_0.tip, _e8p.fdt, _e8p.fdx, _e8p.fnm, _e8p.nvd, _e8p.nvm, _e8p.si, _e8p_Asserting_0.doc, _e8p_Asserting_0.tim, _e8p_Asserting_0.tip, _e8q.fdt, _e8q.fdx, _e8q.fnm, _e8q.nvd, _e8q.nvm, _e8q.si, _e8q_Asserting_0.doc, _e8q_Asserting_0.tim, _e8q_Asserting_0.tip, _e8r.fdt, _e8r.fdx, _e8r.fnm, _e8r.nvd, _e8r.nvm, _e8r.si, _e8r_Asserting_0.doc, _e8r_Asserting_0.tim, _e8r_Asserting_0.tip, _e8s.fdt, _e8s.fdx, _e8s.fnm, _e8s.nvd, _e8s.nvm, _e8s.si, _e8s_Asserting_0.doc, _e8s_Asserting_0.tim, _e8s_Asserting_0.tip, _e8t.fdt, _e8t.fdx, _e8t.fnm, _e8t.nvd, _e8t.nvm, _e8t.si, _e8t_Asserting_0.doc, _e8t_Asserting_0.tim, _e8t_Asserting_0.tip, _e8u.fdt, _e8u.fdx, _e8u.fnm, _e8u.nvd, _e8u.nvm, _e8u.si, _e8u_Asserting_0.doc, _e8u_Asserting_0.tim, _e8u_Asserting_0.tip, _e8v.fdt, _e8v.fdx, _e8v.fnm, _e8v.nvd, _e8v.nvm, _e8v.si, _e8v_Asserting_0.doc, _e8v_Asserting_0.tim, _e8v_Asserting_0.tip, _e8w.fdt, _e8w.fdx, _e8w.fnm, _e8w.nvd, _e8w.nvm, _e8w.si, _e8w_Asserting_0.doc, _e8w_Asserting_0.tim, _e8w_Asserting_0.tip, _e8x.fdt, _e8x.fdx, _e8x.fnm, _e8x.nvd, _e8x.nvm, _e8x.si, _e8x_Asserting_0.doc, _e8x_Asserting_0.tim, _e8x_Asserting_0.tip, _e8y.fdt, _e8y.fdx, _e8y.fnm, _e8y.nvd, _e8y.nvm, _e8y.si, _e8y_Asserting_0.doc, _e8y_Asserting_0.tim, _e8y_Asserting_0.tip, _e8z.fdt, _e8z.fdx, _e8z.fnm, _e8z.nvd, _e8z.nvm, _e8z.si, _e8z_Asserting_0.doc, _e8z_Asserting_0.tim, _e8z_Asserting_0.tip, _e90.fdt, _e90.fdx, _e90.fnm, _e90.nvd, _e90.nvm, _e90.si, _e90_Asserting_0.doc, _e90_Asserting_0.tim, _e90_Asserting_0.tip, _e91.fdt, _e91.fdx, _e91.fnm, _e91.nvd, _e91.nvm, _e91.si, _e91_Asserting_0.doc, _e91_Asserting_0.tim, _e91_Asserting_0.tip, _e92.fdt, _e92.fdx, _e92.fnm, _e92.nvd, _e92.nvm, _e92.si, _e92_Asserting_0.doc, _e92_Asserting_0.tim, _e92_Asserting_0.tip, _e93.fdt, _e93.fdx, _e93.fnm, _e93.nvd, _e93.nvm, _e93.si, _e93_Asserting_0.doc, _e93_Asserting_0.tim, _e93_Asserting_0.tip, _e94.fdt, _e94.fdx, _e94.fnm, _e94.nvd, _e94.nvm, _e94.si, _e94_Asserting_0.doc, _e94_Asserting_0.tim, _e94_Asserting_0.tip, _e95.fdt, _e95.fdx, _e95.fnm, _e95.nvd, _e95.nvm, _e95.si, _e95_Asserting_0.doc, _e95_Asserting_0.tim, _e95_Asserting_0.tip, _e96.fdt, _e96.fdx, _e96.fnm, _e96.nvd, _e96.nvm, _e96.si, _e96_Asserting_0.doc, _e96_Asserting_0.tim, _e96_Asserting_0.tip, _e97.fdt, _e97.fdx, _e97.fnm, _e97.nvd, _e97.nvm, _e97.si, _e97_Asserting_0.doc, _e97_Asserting_0.tim, _e97_Asserting_0.tip, _e98.fdt, _e98.fdx, _e98.fnm, _e98.nvd, _e98.nvm, _e98.si, _e98_Asserting_0.doc, _e98_Asserting_0.tim, _e98_Asserting_0.tip, _e99.fdt, _e99.fdx, _e99.fnm, _e99.nvd, _e99.nvm, _e99.si, _e99_Asserting_0.doc, _e99_Asserting_0.tim, _e99_Asserting_0.tip, _e9a.fdt, _e9a.fdx, _e9a.fnm, _e9a.nvd, _e9a.nvm, _e9a.si, _e9a_Asserting_0.doc, _e9a_Asserting_0.tim, _e9a_Asserting_0.tip, _e9b.fdt, _e9b.fdx, _e9b.fnm, _e9b.nvd, _e9b.nvm, _e9b.si, _e9b_Asserting_0.doc, _e9b_Asserting_0.tim, _e9b_Asserting_0.tip, _e9c.fdt, _e9c.fdx, _e9c.fnm, _e9c.nvd, _e9c.nvm, _e9c.si, _e9c_Asserting_0.doc, _e9c_Asserting_0.tim, _e9c_Asserting_0.tip, _e9d.fdt, _e9d.fdx, _e9d.fnm, _e9d.nvd, _e9d.nvm, _e9d.si, _e9d_Asserting_0.doc, _e9d_Asserting_0.tim, _e9d_Asserting_0.tip, _e9e.fdt, _e9e.fdx, _e9e.fnm, _e9e.nvd, _e9e.nvm, _e9e.si, _e9e_Asserting_0.doc, _e9e_Asserting_0.tim, _e9e_Asserting_0.tip, _e9f.fdt, _e9f.fdx, _e9f.fnm, _e9f.nvd, _e9f.nvm, _e9f.si, _e9f_Asserting_0.doc, _e9f_Asserting_0.tim, _e9f_Asserting_0.tip, _e9g.fdt, _e9g.fdx, _e9g.fnm, _e9g.nvd, _e9g.nvm, _e9g.si, _e9g_Asserting_0.doc, _e9g_Asserting_0.tim, _e9g_Asserting_0.tip, _e9h.fdt, _e9h.fdx, _e9h.fnm, _e9h.nvd, _e9h.nvm, _e9h.si, _e9h_Asserting_0.doc, _e9h_Asserting_0.tim, _e9h_Asserting_0.tip, _edv.fdt, _edv.fdx, _edv.fnm, _edv.nvd, _edv.nvm, _edv.si, _edv_Asserting_0.doc, _edv_Asserting_0.tim, _edv_Asserting_0.tip, _edw.fdt, _edw.fdx, _edw.fnm, _edw.nvd, _edw.nvm, _edw.si, _edw_Asserting_0.doc, _edw_Asserting_0.tim, _edw_Asserting_0.tip, _edx.fdt, _edx.fdx, _edx.fnm, _edx.nvd, _edx.nvm, _edx.si, _edx_Asserting_0.doc, _edx_Asserting_0.tim, _edx_Asserting_0.tip, _edy.fdt, _edy.fdx, _edy.fnm, _edy.nvd, _edy.nvm, _edy.si, _edy_Asserting_0.doc, _edy_Asserting_0.tim, _edy_Asserting_0.tip, _edz.fdt, _edz.fdx, _edz.fnm, _edz.nvd, _edz.nvm, _edz.si, _edz_Asserting_0.doc, _edz_Asserting_0.tim, _edz_Asserting_0.tip, _ee0.fdt, _ee0.fdx, _ee0.fnm, _ee0.nvd, _ee0.nvm, _ee0.si, _ee0_Asserting_0.doc, _ee0_Asserting_0.tim, _ee0_Asserting_0.tip, _ee1.fdt, _ee1.fdx, _ee1.fnm, _ee1.nvd, _ee1.nvm, _ee1.si, _ee1_Asserting_0.doc, _ee1_Asserting_0.tim, _ee1_Asserting_0.tip, _ee2.fdt, _ee2.fdx, _ee2.fnm, _ee2.nvd, _ee2.nvm, _ee2.si, _ee2_Asserting_0.doc, _ee2_Asserting_0.tim, _ee2_Asserting_0.tip, _ee3.fdt, _ee3.fdx, _ee3.fnm, _ee3.nvd, _ee3.nvm, _ee3.si, _ee3_Asserting_0.doc, _ee3_Asserting_0.tim, _ee3_Asserting_0.tip, _ee4.fdt, _ee4.fdx, _ee4.fnm, _ee4.nvd, _ee4.nvm, _ee4.si, _ee4_Asserting_0.doc, _ee4_Asserting_0.tim, _ee4_Asserting_0.tip, _ee5.fdt, _ee5.fdx, _ee5.fnm, _ee5.nvd, _ee5.nvm, _ee5.si, _ee5_Asserting_0.doc, _ee5_Asserting_0.tim, _ee5_Asserting_0.tip, _ee6.fdt, _ee6.fdx, _ee6.fnm, _ee6.nvd, _ee6.nvm, _ee6.si, _ee6_Asserting_0.doc, _ee6_Asserting_0.tim, _ee6_Asserting_0.tip, _ee7.fdt, _ee7.fdx, _ee7.fnm, _ee7.nvd, _ee7.nvm, _ee7.si, _ee7_Asserting_0.doc, _ee7_Asserting_0.tim, _ee7_Asserting_0.tip, _ee8.fdt, _ee8.fdx, _ee8.fnm, _ee8.nvd, _ee8.nvm, _ee8.si, _ee8_Asserting_0.doc, _ee8_Asserting_0.tim, _ee8_Asserting_0.tip, _ee9.fdt, _ee9.fdx, _ee9.fnm, _ee9.nvd, _ee9.nvm, _ee9.si, _ee9_Asserting_0.doc, _ee9_Asserting_0.tim, _ee9_Asserting_0.tip, _eea.fdt, _eea.fdx, _eea.fnm, _eea.nvd, _eea.nvm, _eea.si, _eea_Asserting_0.doc, _eea_Asserting_0.tim, _eea_Asserting_0.tip, _eeb.fdt, _eeb.fdx, _eeb.fnm, _eeb.nvd, _eeb.nvm, _eeb.si, _eeb_Asserting_0.doc, _eeb_Asserting_0.tim, _eeb_Asserting_0.tip, segments_2]}]> but was:<[{indexVersion=1504666109069,generation=2,filelist=[_e35.fdt, _e35.fdx, _e35.fnm, _e35.nvd, _e35.nvm, _e35.si, _e35_Asserting_0.doc, _e35_Asserting_0.tim, _e35_Asserting_0.tip, _e8g.fdt, _e8g.fdx, _e8g.fnm, _e8g.nvd, _e8g.nvm, _e8g.si, _e8g_Asserting_0.doc, _e8g_Asserting_0.tim, _e8g_Asserting_0.tip, _e8h.fdt, _e8h.fdx, _e8h.fnm, _e8h.nvd, _e8h.nvm, _e8h.si, _e8h_Asserting_0.doc, _e8h_Asserting_0.tim, _e8h_Asserting_0.tip, _e8i.fdt, _e8i.fdx, _e8i.fnm, _e8i.nvd, _e8i.nvm, _e8i.si, _e8i_Asserting_0.doc, _e8i_Asserting_0.tim, _e8i_Asserting_0.tip, _e8j.fdt, _e8j.fdx, _e8j.fnm, _e8j.nvd, _e8j.nvm, _e8j.si, _e8j_Asserting_0.doc, _e8j_Asserting_0.tim, _e8j_Asserting_0.tip, _e8k.fdt, _e8k.fdx, _e8k.fnm, _e8k.nvd, _e8k.nvm, _e8k.si, _e8k_Asserting_0.doc, _e8k_Asserting_0.tim, _e8k_Asserting_0.tip, _e8l.fdt, _e8l.fdx, _e8l.fnm, _e8l.nvd, _e8l.nvm, _e8l.si, _e8l_Asserting_0.doc, _e8l_Asserting_0.tim, _e8l_Asserting_0.tip, _e8m.fdt, _e8m.fdx, _e8m.fnm, _e8m.nvd, _e8m.nvm, _e8m.si, _e8m_Asserting_0.doc, _e8m_Asserting_0.tim, _e8m_Asserting_0.tip, _e8n.fdt, _e8n.fdx, _e8n.fnm, _e8n.nvd, _e8n.nvm, _e8n.si, _e8n_Asserting_0.doc, _e8n_Asserting_0.tim, _e8n_Asserting_0.tip, _e8o.fdt, _e8o.fdx, _e8o.fnm, _e8o.nvd, _e8o.nvm, _e8o.si, _e8o_Asserting_0.doc, _e8o_Asserting_0.tim, _e8o_Asserting_0.tip, _e8p.fdt, _e8p.fdx, _e8p.fnm, _e8p.nvd, _e8p.nvm, _e8p.si, _e8p_Asserting_0.doc, _e8p_Asserting_0.tim, _e8p_Asserting_0.tip, _e8q.fdt, _e8q.fdx, _e8q.fnm, _e8q.nvd, _e8q.nvm, _e8q.si, _e8q_Asserting_0.doc, _e8q_Asserting_0.tim, _e8q_Asserting_0.tip, _e8r.fdt, _e8r.fdx, _e8r.fnm, _e8r.nvd, _e8r.nvm, _e8r.si, _e8r_Asserting_0.doc, _e8r_Asserting_0.tim, _e8r_Asserting_0.tip, _e8s.fdt, _e8s.fdx, _e8s.fnm, _e8s.nvd, _e8s.nvm, _e8s.si, _e8s_Asserting_0.doc, _e8s_Asserting_0.tim, _e8s_Asserting_0.tip, _e8t.fdt, _e8t.fdx, _e8t.fnm, _e8t.nvd, _e8t.nvm, _e8t.si, _e8t_Asserting_0.doc, _e8t_Asserting_0.tim, _e8t_Asserting_0.tip, _e8u.fdt, _e8u.fdx, _e8u.fnm, _e8u.nvd, _e8u.nvm, _e8u.si, _e8u_Asserting_0.doc, _e8u_Asserting_0.tim, _e8u_Asserting_0.tip, _e8v.fdt, _e8v.fdx, _e8v.fnm, _e8v.nvd, _e8v.nvm, _e8v.si, _e8v_Asserting_0.doc, _e8v_Asserting_0.tim, _e8v_Asserting_0.tip, _e8w.fdt, _e8w.fdx, _e8w.fnm, _e8w.nvd, _e8w.nvm, _e8w.si, _e8w_Asserting_0.doc, _e8w_Asserting_0.tim, _e8w_Asserting_0.tip, _e8x.fdt, _e8x.fdx, _e8x.fnm, _e8x.nvd, _e8x.nvm, _e8x.si, _e8x_Asserting_0.doc, _e8x_Asserting_0.tim, _e8x_Asserting_0.tip, _e8y.fdt, _e8y.fdx, _e8y.fnm, _e8y.nvd, _e8y.nvm, _e8y.si, _e8y_Asserting_0.doc, _e8y_Asserting_0.tim, _e8y_Asserting_0.tip, _e8z.fdt, _e8z.fdx, _e8z.fnm, _e8z.nvd, _e8z.nvm, _e8z.si, _e8z_Asserting_0.doc, _e8z_Asserting_0.tim, _e8z_Asserting_0.tip, _e90.fdt, _e90.fdx, _e90.fnm, _e90.nvd, _e90.nvm, _e90.si, _e90_Asserting_0.doc, _e90_Asserting_0.tim, _e90_Asserting_0.tip, _e91.fdt, _e91.fdx, _e91.fnm, _e91.nvd, _e91.nvm, _e91.si, _e91_Asserting_0.doc, _e91_Asserting_0.tim, _e91_Asserting_0.tip, _e92.fdt, _e92.fdx, _e92.fnm, _e92.nvd, _e92.nvm, _e92.si, _e92_Asserting_0.doc, _e92_Asserting_0.tim, _e92_Asserting_0.tip, _e93.fdt, _e93.fdx, _e93.fnm, _e93.nvd, _e93.nvm, _e93.si, _e93_Asserting_0.doc, _e93_Asserting_0.tim, _e93_Asserting_0.tip, _e94.fdt, _e94.fdx, _e94.fnm, _e94.nvd, _e94.nvm, _e94.si, _e94_Asserting_0.doc, _e94_Asserting_0.tim, _e94_Asserting_0.tip, _e95.fdt, _e95.fdx, _e95.fnm, _e95.nvd, _e95.nvm, _e95.si, _e95_Asserting_0.doc, _e95_Asserting_0.tim, _e95_Asserting_0.tip, _e96.fdt, _e96.fdx, _e96.fnm, _e96.nvd, _e96.nvm, _e96.si, _e96_Asserting_0.doc, _e96_Asserting_0.tim, _e96_Asserting_0.tip, _e97.fdt, _e97.fdx, _e97.fnm, _e97.nvd, _e97.nvm, _e97.si, _e97_Asserting_0.doc, _e97_Asserting_0.tim, _e97_Asserting_0.tip, _e98.fdt, _e98.fdx, _e98.fnm, _e98.nvd, _e98.nvm, _e98.si, _e98_Asserting_0.doc, _e98_Asserting_0.tim, _e98_Asserting_0.tip, _e99.fdt, _e99.fdx, _e99.fnm, _e99.nvd, _e99.nvm, _e99.si, _e99_Asserting_0.doc, _e99_Asserting_0.tim, _e99_Asserting_0.tip, _e9a.fdt, _e9a.fdx, _e9a.fnm, _e9a.nvd, _e9a.nvm, _e9a.si, _e9a_Asserting_0.doc, _e9a_Asserting_0.tim, _e9a_Asserting_0.tip, _e9b.fdt, _e9b.fdx, _e9b.fnm, _e9b.nvd, _e9b.nvm, _e9b.si, _e9b_Asserting_0.doc, _e9b_Asserting_0.tim, _e9b_Asserting_0.tip, _e9c.fdt, _e9c.fdx, _e9c.fnm, _e9c.nvd, _e9c.nvm, _e9c.si, _e9c_Asserting_0.doc, _e9c_Asserting_0.tim, _e9c_Asserting_0.tip, _e9d.fdt, _e9d.fdx, _e9d.fnm, _e9d.nvd, _e9d.nvm, _e9d.si, _e9d_Asserting_0.doc, _e9d_Asserting_0.tim, _e9d_Asserting_0.tip, _e9e.fdt, _e9e.fdx, _e9e.fnm, _e9e.nvd, _e9e.nvm, _e9e.si, _e9e_Asserting_0.doc, _e9e_Asserting_0.tim, _e9e_Asserting_0.tip, _e9f.fdt, _e9f.fdx, _e9f.fnm, _e9f.nvd, _e9f.nvm, _e9f.si, _e9f_Asserting_0.doc, _e9f_Asserting_0.tim, _e9f_Asserting_0.tip, _e9g.fdt, _e9g.fdx, _e9g.fnm, _e9g.nvd, _e9g.nvm, _e9g.si, _e9g_Asserting_0.doc, _e9g_Asserting_0.tim, _e9g_Asserting_0.tip, _e9h.fdt, _e9h.fdx, _e9h.fnm, _e9h.nvd, _e9h.nvm, _e9h.si, _e9h_Asserting_0.doc, _e9h_Asserting_0.tim, _e9h_Asserting_0.tip, _edv.fdt, _edv.fdx, _edv.fnm, _edv.nvd, _edv.nvm, _edv.si, _edv_Asserting_0.doc, _edv_Asserting_0.tim, _edv_Asserting_0.tip, _edw.fdt, _edw.fdx, _edw.fnm, _edw.nvd, _edw.nvm, _edw.si, _edw_Asserting_0.doc, _edw_Asserting_0.tim, _edw_Asserting_0.tip, _edx.fdt, _edx.fdx, _edx.fnm, _edx.nvd, _edx.nvm, _edx.si, _edx_Asserting_0.doc, _edx_Asserting_0.tim, _edx_Asserting_0.tip, _edy.fdt, _edy.fdx, _edy.fnm, _edy.nvd, _edy.nvm, _edy.si, _edy_Asserting_0.doc, _edy_Asserting_0.tim, _edy_Asserting_0.tip, _edz.fdt, _edz.fdx, _edz.fnm, _edz.nvd, _edz.nvm, _edz.si, _edz_Asserting_0.doc, _edz_Asserting_0.tim, _edz_Asserting_0.tip, _ee0.fdt, _ee0.fdx, _ee0.fnm, _ee0.nvd, _ee0.nvm, _ee0.si, _ee0_Asserting_0.doc, _ee0_Asserting_0.tim, _ee0_Asserting_0.tip, _ee1.fdt, _ee1.fdx, _ee1.fnm, _ee1.nvd, _ee1.nvm, _ee1.si, _ee1_Asserting_0.doc, _ee1_Asserting_0.tim, _ee1_Asserting_0.tip, _ee2.fdt, _ee2.fdx, _ee2.fnm, _ee2.nvd, _ee2.nvm, _ee2.si, _ee2_Asserting_0.doc, _ee2_Asserting_0.tim, _ee2_Asserting_0.tip, _ee3.fdt, _ee3.fdx, _ee3.fnm, _ee3.nvd, _ee3.nvm, _ee3.si, _ee3_Asserting_0.doc, _ee3_Asserting_0.tim, _ee3_Asserting_0.tip, _ee4.fdt, _ee4.fdx, _ee4.fnm, _ee4.nvd, _ee4.nvm, _ee4.si, _ee4_Asserting_0.doc, _ee4_Asserting_0.tim, _ee4_Asserting_0.tip, _ee5.fdt, _ee5.fdx, _ee5.fnm, _ee5.nvd, _ee5.nvm, _ee5.si, _ee5_Asserting_0.doc, _ee5_Asserting_0.tim, _ee5_Asserting_0.tip, _ee6.fdt, _ee6.fdx, _ee6.fnm, _ee6.nvd, _ee6.nvm, _ee6.si, _ee6_Asserting_0.doc, _ee6_Asserting_0.tim, _ee6_Asserting_0.tip, _ee7.fdt, _ee7.fdx, _ee7.fnm, _ee7.nvd, _ee7.nvm, _ee7.si, _ee7_Asserting_0.doc, _ee7_Asserting_0.tim, _ee7_Asserting_0.tip, _ee8.fdt, _ee8.fdx, _ee8.fnm, _ee8.nvd, _ee8.nvm, _ee8.si, _ee8_Asserting_0.doc, _ee8_Asserting_0.tim, _ee8_Asserting_0.tip, _ee9.fdt, _ee9.fdx, _ee9.fnm, _ee9.nvd, _ee9.nvm, _ee9.si, _ee9_Asserting_0.doc, _ee9_Asserting_0.tim, _ee9_Asserting_0.tip, _eea.fdt, _eea.fdx, _eea.fnm, _eea.nvd, _eea.nvm, _eea.si, _eea_Asserting_0.doc, _eea_Asserting_0.tim, _eea_Asserting_0.tip, _eeb.fdt, _eeb.fdx, _eeb.fnm, _eeb.nvd, _eeb.nvm, _eeb.si, _eeb_Asserting_0.doc, _eeb_Asserting_0.tim, _eeb_Asserting_0.tip, segments_2]}, {indexVersion=1504666109069,generation=3,filelist=[_e9j.fdt, _e9j.fdx, _e9j.fnm, _e9j.nvd, _e9j.nvm, _e9j.si, _e9j_Asserting_0.doc, _e9j_Asserting_0.tim, _e9j_Asserting_0.tip, _edv.fdt, _edv.fdx, _edv.fnm, _edv.nvd, _edv.nvm, _edv.si, _edv_Asserting_0.doc, _edv_Asserting_0.tim, _edv_Asserting_0.tip, _edw.fdt, _edw.fdx, _edw.fnm, _edw.nvd, _edw.nvm, _edw.si, _edw_Asserting_0.doc, _edw_Asserting_0.tim, _edw_Asserting_0.tip, _edx.fdt, _edx.fdx, _edx.fnm, _edx.nvd, _edx.nvm, _edx.si, _edx_Asserting_0.doc, _edx_Asserting_0.tim, _edx_Asserting_0.tip, _edy.fdt, _edy.fdx, _edy.fnm, _edy.nvd, _edy.nvm, _edy.si, _edy_Asserting_0.doc, _edy_Asserting_0.tim, _edy_Asserting_0.tip, _edz.fdt, _edz.fdx, _edz.fnm, _edz.nvd, _edz.nvm, _edz.si, _edz_Asserting_0.doc, _edz_Asserting_0.tim, _edz_Asserting_0.tip, _ee0.fdt, _ee0.fdx, _ee0.fnm, _ee0.nvd, _ee0.nvm, _ee0.si, _ee0_Asserting_0.doc, _ee0_Asserting_0.tim, _ee0_Asserting_0.tip, _ee1.fdt, _ee1.fdx, _ee1.fnm, _ee1.nvd, _ee1.nvm, _ee1.si, _ee1_Asserting_0.doc, _ee1_Asserting_0.tim, _ee1_Asserting_0.tip, _ee2.fdt, _ee2.fdx, _ee2.fnm, _ee2.nvd, _ee2.nvm, _ee2.si, _ee2_Asserting_0.doc, _ee2_Asserting_0.tim, _ee2_Asserting_0.tip, _ee3.fdt, _ee3.fdx, _ee3.fnm, _ee3.nvd, _ee3.nvm, _ee3.si, _ee3_Asserting_0.doc, _ee3_Asserting_0.tim, _ee3_Asserting_0.tip, _ee4.fdt, _ee4.fdx, _ee4.fnm, _ee4.nvd, _ee4.nvm, _ee4.si, _ee4_Asserting_0.doc, _ee4_Asserting_0.tim, _ee4_Asserting_0.tip, _ee5.fdt, _ee5.fdx, _ee5.fnm, _ee5.nvd, _ee5.nvm, _ee5.si, _ee5_Asserting_0.doc, _ee5_Asserting_0.tim, _ee5_Asserting_0.tip, _ee6.fdt, _ee6.fdx, _ee6.fnm, _ee6.nvd, _ee6.nvm, _ee6.si, _ee6_Asserting_0.doc, _ee6_Asserting_0.tim, _ee6_Asserting_0.tip, _ee7.fdt, _ee7.fdx, _ee7.fnm, _ee7.nvd, _ee7.nvm, _ee7.si, _ee7_Asserting_0.doc, _ee7_Asserting_0.tim, _ee7_Asserting_0.tip, _ee8.fdt, _ee8.fdx, _ee8.fnm, _ee8.nvd, _ee8.nvm, _ee8.si, _ee8_Asserting_0.doc, _ee8_Asserting_0.tim, _ee8_Asserting_0.tip, _ee9.fdt, _ee9.fdx, _ee9.fnm, _ee9.nvd, _ee9.nvm, _ee9.si, _ee9_Asserting_0.doc, _ee9_Asserting_0.tim, _ee9_Asserting_0.tip, _eea.fdt, _eea.fdx, _eea.fnm, _eea.nvd, _eea.nvm, _eea.si, _eea_Asserting_0.doc, _eea_Asserting_0.tim, _eea_Asserting_0.tip, _eeb.fdt, _eeb.fdx, _eeb.fnm, _eeb.nvd, _eeb.nvm, _eeb.si, _eeb_Asserting_0.doc, _eeb_Asserting_0.tim, _eeb_Asserting_0.tip, segments_3]}]>
Stack Trace:
java.lang.AssertionError: expected:<[{indexVersion=1504666109069,generation=2,filelist=[_e35.fdt, _e35.fdx, _e35.fnm, _e35.nvd, _e35.nvm, _e35.si, _e35_Asserting_0.doc, _e35_Asserting_0.tim, _e35_Asserting_0.tip, _e8g.fdt, _e8g.fdx, _e8g.fnm, _e8g.nvd, _e8g.nvm, _e8g.si, _e8g_Asserting_0.doc, _e8g_Asserting_0.tim, _e8g_Asserting_0.tip, _e8h.fdt, _e8h.fdx, _e8h.fnm, _e8h.nvd, _e8h.nvm, _e8h.si, _e8h_Asserting_0.doc, _e8h_Asserting_0.tim, _e8h_Asserting_0.tip, _e8i.fdt, _e8i.fdx, _e8i.fnm, _e8i.nvd, _e8i.nvm, _e8i.si, _e8i_Asserting_0.doc, _e8i_Asserting_0.tim, _e8i_Asserting_0.tip, _e8j.fdt, _e8j.fdx, _e8j.fnm, _e8j.nvd, _e8j.nvm, _e8j.si, _e8j_Asserting_0.doc, _e8j_Asserting_0.tim, _e8j_Asserting_0.tip, _e8k.fdt, _e8k.fdx, _e8k.fnm, _e8k.nvd, _e8k.nvm, _e8k.si, _e8k_Asserting_0.doc, _e8k_Asserting_0.tim, _e8k_Asserting_0.tip, _e8l.fdt, _e8l.fdx, _e8l.fnm, _e8l.nvd, _e8l.nvm, _e8l.si, _e8l_Asserting_0.doc, _e8l_Asserting_0.tim, _e8l_Asserting_0.tip, _e8m.fdt, _e8m.fdx, _e8m.fnm, _e8m.nvd, _e8m.nvm, _e8m.si, _e8m_Asserting_0.doc, _e8m_Asserting_0.tim, _e8m_Asserting_0.tip, _e8n.fdt, _e8n.fdx, _e8n.fnm, _e8n.nvd, _e8n.nvm, _e8n.si, _e8n_Asserting_0.doc, _e8n_Asserting_0.tim, _e8n_Asserting_0.tip, _e8o.fdt, _e8o.fdx, _e8o.fnm, _e8o.nvd, _e8o.nvm, _e8o.si, _e8o_Asserting_0.doc, _e8o_Asserting_0.tim, _e8o_Asserting_0.tip, _e8p.fdt, _e8p.fdx, _e8p.fnm, _e8p.nvd, _e8p.nvm, _e8p.si, _e8p_Asserting_0.doc, _e8p_Asserting_0.tim, _e8p_Asserting_0.tip, _e8q.fdt, _e8q.fdx, _e8q.fnm, _e8q.nvd, _e8q.nvm, _e8q.si, _e8q_Asserting_0.doc, _e8q_Asserting_0.tim, _e8q_Asserting_0.tip, _e8r.fdt, _e8r.fdx, _e8r.fnm, _e8r.nvd, _e8r.nvm, _e8r.si, _e8r_Asserting_0.doc, _e8r_Asserting_0.tim, _e8r_Asserting_0.tip, _e8s.fdt, _e8s.fdx, _e8s.fnm, _e8s.nvd, _e8s.nvm, _e8s.si, _e8s_Asserting_0.doc, _e8s_Asserting_0.tim, _e8s_Asserting_0.tip, _e8t.fdt, _e8t.fdx, _e8t.fnm, _e8t.nvd, _e8t.nvm, _e8t.si, _e8t_Asserting_0.doc, _e8t_Asserting_0.tim, _e8t_Asserting_0.tip, _e8u.fdt, _e8u.fdx, _e8u.fnm, _e8u.nvd, _e8u.nvm, _e8u.si, _e8u_Asserting_0.doc, _e8u_Asserting_0.tim, _e8u_Asserting_0.tip, _e8v.fdt, _e8v.fdx, _e8v.fnm, _e8v.nvd, _e8v.nvm, _e8v.si, _e8v_Asserting_0.doc, _e8v_Asserting_0.tim, _e8v_Asserting_0.tip, _e8w.fdt, _e8w.fdx, _e8w.fnm, _e8w.nvd, _e8w.nvm, _e8w.si, _e8w_Asserting_0.doc, _e8w_Asserting_0.tim, _e8w_Asserting_0.tip, _e8x.fdt, _e8x.fdx, _e8x.fnm, _e8x.nvd, _e8x.nvm, _e8x.si, _e8x_Asserting_0.doc, _e8x_Asserting_0.tim, _e8x_Asserting_0.tip, _e8y.fdt, _e8y.fdx, _e8y.fnm, _e8y.nvd, _e8y.nvm, _e8y.si, _e8y_Asserting_0.doc, _e8y_Asserting_0.tim, _e8y_Asserting_0.tip, _e8z.fdt, _e8z.fdx, _e8z.fnm, _e8z.nvd, _e8z.nvm, _e8z.si, _e8z_Asserting_0.doc, _e8z_Asserting_0.tim, _e8z_Asserting_0.tip, _e90.fdt, _e90.fdx, _e90.fnm, _e90.nvd, _e90.nvm, _e90.si, _e90_Asserting_0.doc, _e90_Asserting_0.tim, _e90_Asserting_0.tip, _e91.fdt, _e91.fdx, _e91.fnm, _e91.nvd, _e91.nvm, _e91.si, _e91_Asserting_0.doc, _e91_Asserting_0.tim, _e91_Asserting_0.tip, _e92.fdt, _e92.fdx, _e92.fnm, _e92.nvd, _e92.nvm, _e92.si, _e92_Asserting_0.doc, _e92_Asserting_0.tim, _e92_Asserting_0.tip, _e93.fdt, _e93.fdx, _e93.fnm, _e93.nvd, _e93.nvm, _e93.si, _e93_Asserting_0.doc, _e93_Asserting_0.tim, _e93_Asserting_0.tip, _e94.fdt, _e94.fdx, _e94.fnm, _e94.nvd, _e94.nvm, _e94.si, _e94_Asserting_0.doc, _e94_Asserting_0.tim, _e94_Asserting_0.tip, _e95.fdt, _e95.fdx, _e95.fnm, _e95.nvd, _e95.nvm, _e95.si, _e95_Asserting_0.doc, _e95_Asserting_0.tim, _e95_Asserting_0.tip, _e96.fdt, _e96.fdx, _e96.fnm, _e96.nvd, _e96.nvm, _e96.si, _e96_Asserting_0.doc, _e96_Asserting_0.tim, _e96_Asserting_0.tip, _e97.fdt, _e97.fdx, _e97.fnm, _e97.nvd, _e97.nvm, _e97.si, _e97_Asserting_0.doc, _e97_Asserting_0.tim, _e97_Asserting_0.tip, _e98.fdt, _e98.fdx, _e98.fnm, _e98.nvd, _e98.nvm, _e98.si, _e98_Asserting_0.doc, _e98_Asserting_0.tim, _e98_Asserting_0.tip, _e99.fdt, _e99.fdx, _e99.fnm, _e99.nvd, _e99.nvm, _e99.si, _e99_Asserting_0.doc, _e99_Asserting_0.tim, _e99_Asserting_0.tip, _e9a.fdt, _e9a.fdx, _e9a.fnm, _e9a.nvd, _e9a.nvm, _e9a.si, _e9a_Asserting_0.doc, _e9a_Asserting_0.tim, _e9a_Asserting_0.tip, _e9b.fdt, _e9b.fdx, _e9b.fnm, _e9b.nvd, _e9b.nvm, _e9b.si, _e9b_Asserting_0.doc, _e9b_Asserting_0.tim, _e9b_Asserting_0.tip, _e9c.fdt, _e9c.fdx, _e9c.fnm, _e9c.nvd, _e9c.nvm, _e9c.si, _e9c_Asserting_0.doc, _e9c_Asserting_0.tim, _e9c_Asserting_0.tip, _e9d.fdt, _e9d.fdx, _e9d.fnm, _e9d.nvd, _e9d.nvm, _e9d.si, _e9d_Asserting_0.doc, _e9d_Asserting_0.tim, _e9d_Asserting_0.tip, _e9e.fdt, _e9e.fdx, _e9e.fnm, _e9e.nvd, _e9e.nvm, _e9e.si, _e9e_Asserting_0.doc, _e9e_Asserting_0.tim, _e9e_Asserting_0.tip, _e9f.fdt, _e9f.fdx, _e9f.fnm, _e9f.nvd, _e9f.nvm, _e9f.si, _e9f_Asserting_0.doc, _e9f_Asserting_0.tim, _e9f_Asserting_0.tip, _e9g.fdt, _e9g.fdx, _e9g.fnm, _e9g.nvd, _e9g.nvm, _e9g.si, _e9g_Asserting_0.doc, _e9g_Asserting_0.tim, _e9g_Asserting_0.tip, _e9h.fdt, _e9h.fdx, _e9h.fnm, _e9h.nvd, _e9h.nvm, _e9h.si, _e9h_Asserting_0.doc, _e9h_Asserting_0.tim, _e9h_Asserting_0.tip, _edv.fdt, _edv.fdx, _edv.fnm, _edv.nvd, _edv.nvm, _edv.si, _edv_Asserting_0.doc, _edv_Asserting_0.tim, _edv_Asserting_0.tip, _edw.fdt, _edw.fdx, _edw.fnm, _edw.nvd, _edw.nvm, _edw.si, _edw_Asserting_0.doc, _edw_Asserting_0.tim, _edw_Asserting_0.tip, _edx.fdt, _edx.fdx, _edx.fnm, _edx.nvd, _edx.nvm, _edx.si, _edx_Asserting_0.doc, _edx_Asserting_0.tim, _edx_Asserting_0.tip, _edy.fdt, _edy.fdx, _edy.fnm, _edy.nvd, _edy.nvm, _edy.si, _edy_Asserting_0.doc, _edy_Asserting_0.tim, _edy_Asserting_0.tip, _edz.fdt, _edz.fdx, _edz.fnm, _edz.nvd, _edz.nvm, _edz.si, _edz_Asserting_0.doc, _edz_Asserting_0.tim, _edz_Asserting_0.tip, _ee0.fdt, _ee0.fdx, _ee0.fnm, _ee0.nvd, _ee0.nvm, _ee0.si, _ee0_Asserting_0.doc, _ee0_Asserting_0.tim, _ee0_Asserting_0.tip, _ee1.fdt, _ee1.fdx, _ee1.fnm, _ee1.nvd, _ee1.nvm, _ee1.si, _ee1_Asserting_0.doc, _ee1_Asserting_0.tim, _ee1_Asserting_0.tip, _ee2.fdt, _ee2.fdx, _ee2.fnm, _ee2.nvd, _ee2.nvm, _ee2.si, _ee2_Asserting_0.doc, _ee2_Asserting_0.tim, _ee2_Asserting_0.tip, _ee3.fdt, _ee3.fdx, _ee3.fnm, _ee3.nvd, _ee3.nvm, _ee3.si, _ee3_Asserting_0.doc, _ee3_Asserting_0.tim, _ee3_Asserting_0.tip, _ee4.fdt, _ee4.fdx, _ee4.fnm, _ee4.nvd, _ee4.nvm, _ee4.si, _ee4_Asserting_0.doc, _ee4_Asserting_0.tim, _ee4_Asserting_0.tip, _ee5.fdt, _ee5.fdx, _ee5.fnm, _ee5.nvd, _ee5.nvm, _ee5.si, _ee5_Asserting_0.doc, _ee5_Asserting_0.tim, _ee5_Asserting_0.tip, _ee6.fdt, _ee6.fdx, _ee6.fnm, _ee6.nvd, _ee6.nvm, _ee6.si, _ee6_Asserting_0.doc, _ee6_Asserting_0.tim, _ee6_Asserting_0.tip, _ee7.fdt, _ee7.fdx, _ee7.fnm, _ee7.nvd, _ee7.nvm, _ee7.si, _ee7_Asserting_0.doc, _ee7_Asserting_0.tim, _ee7_Asserting_0.tip, _ee8.fdt, _ee8.fdx, _ee8.fnm, _ee8.nvd, _ee8.nvm, _ee8.si, _ee8_Asserting_0.doc, _ee8_Asserting_0.tim, _ee8_Asserting_0.tip, _ee9.fdt, _ee9.fdx, _ee9.fnm, _ee9.nvd, _ee9.nvm, _ee9.si, _ee9_Asserting_0.doc, _ee9_Asserting_0.tim, _ee9_Asserting_0.tip, _eea.fdt, _eea.fdx, _eea.fnm, _eea.nvd, _eea.nvm, _eea.si, _eea_Asserting_0.doc, _eea_Asserting_0.tim, _eea_Asserting_0.tip, _eeb.fdt, _eeb.fdx, _eeb.fnm, _eeb.nvd, _eeb.nvm, _eeb.si, _eeb_Asserting_0.doc, _eeb_Asserting_0.tim, _eeb_Asserting_0.tip, segments_2]}]> but was:<[{indexVersion=1504666109069,generation=2,filelist=[_e35.fdt, _e35.fdx, _e35.fnm, _e35.nvd, _e35.nvm, _e35.si, _e35_Asserting_0.doc, _e35_Asserting_0.tim, _e35_Asserting_0.tip, _e8g.fdt, _e8g.fdx, _e8g.fnm, _e8g.nvd, _e8g.nvm, _e8g.si, _e8g_Asserting_0.doc, _e8g_Asserting_0.tim, _e8g_Asserting_0.tip, _e8h.fdt, _e8h.fdx, _e8h.fnm, _e8h.nvd, _e8h.nvm, _e8h.si, _e8h_Asserting_0.doc, _e8h_Asserting_0.tim, _e8h_Asserting_0.tip, _e8i.fdt, _e8i.fdx, _e8i.fnm, _e8i.nvd, _e8i.nvm, _e8i.si, _e8i_Asserting_0.doc, _e8i_Asserting_0.tim, _e8i_Asserting_0.tip, _e8j.fdt, _e8j.fdx, _e8j.fnm, _e8j.nvd, _e8j.nvm, _e8j.si, _e8j_Asserting_0.doc, _e8j_Asserting_0.tim, _e8j_Asserting_0.tip, _e8k.fdt, _e8k.fdx, _e8k.fnm, _e8k.nvd, _e8k.nvm, _e8k.si, _e8k_Asserting_0.doc, _e8k_Asserting_0.tim, _e8k_Asserting_0.tip, _e8l.fdt, _e8l.fdx, _e8l.fnm, _e8l.nvd, _e8l.nvm, _e8l.si, _e8l_Asserting_0.doc, _e8l_Asserting_0.tim, _e8l_Asserting_0.tip, _e8m.fdt, _e8m.fdx, _e8m.fnm, _e8m.nvd, _e8m.nvm, _e8m.si, _e8m_Asserting_0.doc, _e8m_Asserting_0.tim, _e8m_Asserting_0.tip, _e8n.fdt, _e8n.fdx, _e8n.fnm, _e8n.nvd, _e8n.nvm, _e8n.si, _e8n_Asserting_0.doc, _e8n_Asserting_0.tim, _e8n_Asserting_0.tip, _e8o.fdt, _e8o.fdx, _e8o.fnm, _e8o.nvd, _e8o.nvm, _e8o.si, _e8o_Asserting_0.doc, _e8o_Asserting_0.tim, _e8o_Asserting_0.tip, _e8p.fdt, _e8p.fdx, _e8p.fnm, _e8p.nvd, _e8p.nvm, _e8p.si, _e8p_Asserting_0.doc, _e8p_Asserting_0.tim, _e8p_Asserting_0.tip, _e8q.fdt, _e8q.fdx, _e8q.fnm, _e8q.nvd, _e8q.nvm, _e8q.si, _e8q_Asserting_0.doc, _e8q_Asserting_0.tim, _e8q_Asserting_0.tip, _e8r.fdt, _e8r.fdx, _e8r.fnm, _e8r.nvd, _e8r.nvm, _e8r.si, _e8r_Asserting_0.doc, _e8r_Asserting_0.tim, _e8r_Asserting_0.tip, _e8s.fdt, _e8s.fdx, _e8s.fnm, _e8s.nvd, _e8s.nvm, _e8s.si, _e8s_Asserting_0.doc, _e8s_Asserting_0.tim, _e8s_Asserting_0.tip, _e8t.fdt, _e8t.fdx, _e8t.fnm, _e8t.nvd, _e8t.nvm, _e8t.si, _e8t_Asserting_0.doc, _e8t_Asserting_0.tim, _e8t_Asserting_0.tip, _e8u.fdt, _e8u.fdx, _e8u.fnm, _e8u.nvd, _e8u.nvm, _e8u.si, _e8u_Asserting_0.doc, _e8u_Asserting_0.tim, _e8u_Asserting_0.tip, _e8v.fdt, _e8v.fdx, _e8v.fnm, _e8v.nvd, _e8v.nvm, _e8v.si, _e8v_Asserting_0.doc, _e8v_Asserting_0.tim, _e8v_Asserting_0.tip, _e8w.fdt, _e8w.fdx, _e8w.fnm, _e8w.nvd, _e8w.nvm, _e8w.si, _e8w_Asserting_0.doc, _e8w_Asserting_0.tim, _e8w_Asserting_0.tip, _e8x.fdt, _e8x.fdx, _e8x.fnm, _e8x.nvd, _e8x.nvm, _e8x.si, _e8x_Asserting_0.doc, _e8x_Asserting_0.tim, _e8x_Asserting_0.tip, _e8y.fdt, _e8y.fdx, _e8y.fnm, _e8y.nvd, _e8y.nvm, _e8y.si, _e8y_Asserting_0.doc, _e8y_Asserting_0.tim, _e8y_Asserting_0.tip, _e8z.fdt, _e8z.fdx, _e8z.fnm, _e8z.nvd, _e8z.nvm, _e8z.si, _e8z_Asserting_0.doc, _e8z_Asserting_0.tim, _e8z_Asserting_0.tip, _e90.fdt, _e90.fdx, _e90.fnm, _e90.nvd, _e90.nvm, _e90.si, _e90_Asserting_0.doc, _e90_Asserting_0.tim, _e90_Asserting_0.tip, _e91.fdt, _e91.fdx, _e91.fnm, _e91.nvd, _e91.nvm, _e91.si, _e91_Asserting_0.doc, _e91_Asserting_0.tim, _e91_Asserting_0.tip, _e92.fdt, _e92.fdx, _e92.fnm, _e92.nvd, _e92.nvm, _e92.si, _e92_Asserting_0.doc, _e92_Asserting_0.tim, _e92_Asserting_0.tip, _e93.fdt, _e93.fdx, _e93.fnm, _e93.nvd, _e93.nvm, _e93.si, _e93_Asserting_0.doc, _e93_Asserting_0.tim, _e93_Asserting_0.tip, _e94.fdt, _e94.fdx, _e94.fnm, _e94.nvd, _e94.nvm, _e94.si, _e94_Asserting_0.doc, _e94_Asserting_0.tim, _e94_Asserting_0.tip, _e95.fdt, _e95.fdx, _e95.fnm, _e95.nvd, _e95.nvm, _e95.si, _e95_Asserting_0.doc, _e95_Asserting_0.tim, _e95_Asserting_0.tip, _e96.fdt, _e96.fdx, _e96.fnm, _e96.nvd, _e96.nvm, _e96.si, _e96_Asserting_0.doc, _e96_Asserting_0.tim, _e96_Asserting_0.tip, _e97.fdt, _e97.fdx, _e97.fnm, _e97.nvd, _e97.nvm, _e97.si, _e97_Asserting_0.doc, _e97_Asserting_0.tim, _e97_Asserting_0.tip, _e98.fdt, _e98.fdx, _e98.fnm, _e98.nvd, _e98.nvm, _e98.si, _e98_Asserting_0.doc, _e98_Asserting_0.tim, _e98_Asserting_0.tip, _e99.fdt, _e99.fdx, _e99.fnm, _e99.nvd, _e99.nvm, _e99.si, _e99_Asserting_0.doc, _e99_Asserting_0.tim, _e99_Asserting_0.tip, _e9a.fdt, _e9a.fdx, _e9a.fnm, _e9a.nvd, _e9a.nvm, _e9a.si, _e9a_Asserting_0.doc, _e9a_Asserting_0.tim, _e9a_Asserting_0.tip, _e9b.fdt, _e9b.fdx, _e9b.fnm, _e9b.nvd, _e9b.nvm, _e9b.si, _e9b_Asserting_0.doc, _e9b_Asserting_0.tim, _e9b_Asserting_0.tip, _e9c.fdt, _e9c.fdx, _e9c.fnm, _e9c.nvd, _e9c.nvm, _e9c.si, _e9c_Asserting_0.doc, _e9c_Asserting_0.tim, _e9c_Asserting_0.tip, _e9d.fdt, _e9d.fdx, _e9d.fnm, _e9d.nvd, _e9d.nvm, _e9d.si, _e9d_Asserting_0.doc, _e9d_Asserting_0.tim, _e9d_Asserting_0.tip, _e9e.fdt, _e9e.fdx, _e9e.fnm, _e9e.nvd, _e9e.nvm, _e9e.si, _e9e_Asserting_0.doc, _e9e_Asserting_0.tim, _e9e_Asserting_0.tip, _e9f.fdt, _e9f.fdx, _e9f.fnm, _e9f.nvd, _e9f.nvm, _e9f.si, _e9f_Asserting_0.doc, _e9f_Asserting_0.tim, _e9f_Asserting_0.tip, _e9g.fdt, _e9g.fdx, _e9g.fnm, _e9g.nvd, _e9g.nvm, _e9g.si, _e9g_Asserting_0.doc, _e9g_Asserting_0.tim, _e9g_Asserting_0.tip, _e9h.fdt, _e9h.fdx, _e9h.fnm, _e9h.nvd, _e9h.nvm, _e9h.si, _e9h_Asserting_0.doc, _e9h_Asserting_0.tim, _e9h_Asserting_0.tip, _edv.fdt, _edv.fdx, _edv.fnm, _edv.nvd, _edv.nvm, _edv.si, _edv_Asserting_0.doc, _edv_Asserting_0.tim, _edv_Asserting_0.tip, _edw.fdt, _edw.fdx, _edw.fnm, _edw.nvd, _edw.nvm, _edw.si, _edw_Asserting_0.doc, _edw_Asserting_0.tim, _edw_Asserting_0.tip, _edx.fdt, _edx.fdx, _edx.fnm, _edx.nvd, _edx.nvm, _edx.si, _edx_Asserting_0.doc, _edx_Asserting_0.tim, _edx_Asserting_0.tip, _edy.fdt, _edy.fdx, _edy.fnm, _edy.nvd, _edy.nvm, _edy.si, _edy_Asserting_0.doc, _edy_Asserting_0.tim, _edy_Asserting_0.tip, _edz.fdt, _edz.fdx, _edz.fnm, _edz.nvd, _edz.nvm, _edz.si, _edz_Asserting_0.doc, _edz_Asserting_0.tim, _edz_Asserting_0.tip, _ee0.fdt, _ee0.fdx, _ee0.fnm, _ee0.nvd, _ee0.nvm, _ee0.si, _ee0_Asserting_0.doc, _ee0_Asserting_0.tim, _ee0_Asserting_0.tip, _ee1.fdt, _ee1.fdx, _ee1.fnm, _ee1.nvd, _ee1.nvm, _ee1.si, _ee1_Asserting_0.doc, _ee1_Asserting_0.tim, _ee1_Asserting_0.tip, _ee2.fdt, _ee2.fdx, _ee2.fnm, _ee2.nvd, _ee2.nvm, _ee2.si, _ee2_Asserting_0.doc, _ee2_Asserting_0.tim, _ee2_Asserting_0.tip, _ee3.fdt, _ee3.fdx, _ee3.fnm, _ee3.nvd, _ee3.nvm, _ee3.si, _ee3_Asserting_0.doc, _ee3_Asserting_0.tim, _ee3_Asserting_0.tip, _ee4.fdt, _ee4.fdx, _ee4.fnm, _ee4.nvd, _ee4.nvm, _ee4.si, _ee4_Asserting_0.doc, _ee4_Asserting_0.tim, _ee4_Asserting_0.tip, _ee5.fdt, _ee5.fdx, _ee5.fnm, _ee5.nvd, _ee5.nvm, _ee5.si, _ee5_Asserting_0.doc, _ee5_Asserting_0.tim, _ee5_Asserting_0.tip, _ee6.fdt, _ee6.fdx, _ee6.fnm, _ee6.nvd, _ee6.nvm, _ee6.si, _ee6_Asserting_0.doc, _ee6_Asserting_0.tim, _ee6_Asserting_0.tip, _ee7.fdt, _ee7.fdx, _ee7.fnm, _ee7.nvd, _ee7.nvm, _ee7.si, _ee7_Asserting_0.doc, _ee7_Asserting_0.tim, _ee7_Asserting_0.tip, _ee8.fdt, _ee8.fdx, _ee8.fnm, _ee8.nvd, _ee8.nvm, _ee8.si, _ee8_Asserting_0.doc, _ee8_Asserting_0.tim, _ee8_Asserting_0.tip, _ee9.fdt, _ee9.fdx, _ee9.fnm, _ee9.nvd, _ee9.nvm, _ee9.si, _ee9_Asserting_0.doc, _ee9_Asserting_0.tim, _ee9_Asserting_0.tip, _eea.fdt, _eea.fdx, _eea.fnm, _eea.nvd, _eea.nvm, _eea.si, _eea_Asserting_0.doc, _eea_Asserting_0.tim, _eea_Asserting_0.tip, _eeb.fdt, _eeb.fdx, _eeb.fnm, _eeb.nvd, _eeb.nvm, _eeb.si, _eeb_Asserting_0.doc, _eeb_Asserting_0.tim, _eeb_Asserting_0.tip, segments_2]}, {indexVersion=1504666109069,generation=3,filelist=[_e9j.fdt, _e9j.fdx, _e9j.fnm, _e9j.nvd, _e9j.nvm, _e9j.si, _e9j_Asserting_0.doc, _e9j_Asserting_0.tim, _e9j_Asserting_0.tip, _edv.fdt, _edv.fdx, _edv.fnm, _edv.nvd, _edv.nvm, _edv.si, _edv_Asserting_0.doc, _edv_Asserting_0.tim, _edv_Asserting_0.tip, _edw.fdt, _edw.fdx, _edw.fnm, _edw.nvd, _edw.nvm, _edw.si, _edw_Asserting_0.doc, _edw_Asserting_0.tim, _edw_Asserting_0.tip, _edx.fdt, _edx.fdx, _edx.fnm, _edx.nvd, _edx.nvm, _edx.si, _edx_Asserting_0.doc, _edx_Asserting_0.tim, _edx_Asserting_0.tip, _edy.fdt, _edy.fdx, _edy.fnm, _edy.nvd, _edy.nvm, _edy.si, _edy_Asserting_0.doc, _edy_Asserting_0.tim, _edy_Asserting_0.tip, _edz.fdt, _edz.fdx, _edz.fnm, _edz.nvd, _edz.nvm, _edz.si, _edz_Asserting_0.doc, _edz_Asserting_0.tim, _edz_Asserting_0.tip, _ee0.fdt, _ee0.fdx, _ee0.fnm, _ee0.nvd, _ee0.nvm, _ee0.si, _ee0_Asserting_0.doc, _ee0_Asserting_0.tim, _ee0_Asserting_0.tip, _ee1.fdt, _ee1.fdx, _ee1.fnm, _ee1.nvd, _ee1.nvm, _ee1.si, _ee1_Asserting_0.doc, _ee1_Asserting_0.tim, _ee1_Asserting_0.tip, _ee2.fdt, _ee2.fdx, _ee2.fnm, _ee2.nvd, _ee2.nvm, _ee2.si, _ee2_Asserting_0.doc, _ee2_Asserting_0.tim, _ee2_Asserting_0.tip, _ee3.fdt, _ee3.fdx, _ee3.fnm, _ee3.nvd, _ee3.nvm, _ee3.si, _ee3_Asserting_0.doc, _ee3_Asserting_0.tim, _ee3_Asserting_0.tip, _ee4.fdt, _ee4.fdx, _ee4.fnm, _ee4.nvd, _ee4.nvm, _ee4.si, _ee4_Asserting_0.doc, _ee4_Asserting_0.tim, _ee4_Asserting_0.tip, _ee5.fdt, _ee5.fdx, _ee5.fnm, _ee5.nvd, _ee5.nvm, _ee5.si, _ee5_Asserting_0.doc, _ee5_Asserting_0.tim, _ee5_Asserting_0.tip, _ee6.fdt, _ee6.fdx, _ee6.fnm, _ee6.nvd, _ee6.nvm, _ee6.si, _ee6_Asserting_0.doc, _ee6_Asserting_0.tim, _ee6_Asserting_0.tip, _ee7.fdt, _ee7.fdx, _ee7.fnm, _ee7.nvd, _ee7.nvm, _ee7.si, _ee7_Asserting_0.doc, _ee7_Asserting_0.tim, _ee7_Asserting_0.tip, _ee8.fdt, _ee8.fdx, _ee8.fnm, _ee8.nvd, _ee8.nvm, _ee8.si, _ee8_Asserting_0.doc, _ee8_Asserting_0.tim, _ee8_Asserting_0.tip, _ee9.fdt, _ee9.fdx, _ee9.fnm, _ee9.nvd, _ee9.nvm, _ee9.si, _ee9_Asserting_0.doc, _ee9_Asserting_0.tim, _ee9_Asserting_0.tip, _eea.fdt, _eea.fdx, _eea.fnm, _eea.nvd, _eea.nvm, _eea.si, _eea_Asserting_0.doc, _eea_Asserting_0.tim, _eea_Asserting_0.tip, _eeb.fdt, _eeb.fdx, _eeb.fnm, _eeb.nvd, _eeb.nvm, _eeb.si, _eeb_Asserting_0.doc, _eeb_Asserting_0.tim, _eeb_Asserting_0.tip, segments_3]}]>
at __randomizedtesting.SeedInfo.seed([A85C3D2AC3D0CB06:8D8B261AB398C505]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:147)
at org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload(TestReplicationHandler.java:1277)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.client.solrj.TestLBHttpSolrClient.testSimple
Error Message:
expected:<3> but was:<2>
Stack Trace:
java.lang.AssertionError: expected:<3> but was:<2>
at __randomizedtesting.SeedInfo.seed([2FF29CB3A774AC20:1741B84D808778F1]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.client.solrj.TestLBHttpSolrClient.testSimple(TestLBHttpSolrClient.java:170)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test
Error Message:
expected:<2> but was:<3>
Stack Trace:
java.lang.AssertionError: expected:<2> but was:<3>
at __randomizedtesting.SeedInfo.seed([A85C3D2AC3D0CB06:200802F06D2CA6FE]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.assertSliceAndReplicaCount(SharedFSAutoReplicaFailoverTest.java:403)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.testBasics(SharedFSAutoReplicaFailoverTest.java:248)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test(SharedFSAutoReplicaFailoverTest.java:136)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12181 lines...]
[junit4] Suite: org.apache.solr.handler.TestReplicationHandler
[junit4] 2> 579703 INFO (SUITE-TestReplicationHandler-seed#[A85C3D2AC3D0CB06]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A85C3D2AC3D0CB06-001/init-core-data-001
[junit4] 2> 579703 WARN (SUITE-TestReplicationHandler-seed#[A85C3D2AC3D0CB06]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=2 numCloses=2
[junit4] 2> 579703 INFO (SUITE-TestReplicationHandler-seed#[A85C3D2AC3D0CB06]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 579705 INFO (SUITE-TestReplicationHandler-seed#[A85C3D2AC3D0CB06]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=None)
[junit4] 2> 579712 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[A85C3D2AC3D0CB06]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testEmptyCommits
[junit4] 2> 579712 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[A85C3D2AC3D0CB06]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A85C3D2AC3D0CB06-001/solr-instance-001/collection1
[junit4] 2> 579715 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[A85C3D2AC3D0CB06]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 579830 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[A85C3D2AC3D0CB06]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@20753110{/solr,null,AVAILABLE}
[junit4] 2> 579831 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[A85C3D2AC3D0CB06]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7ada187{HTTP/1.1,[http/1.1]}{127.0.0.1:48638}
[junit4] 2> 579831 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[A85C3D2AC3D0CB06]) [ ] o.e.j.s.Server Started @585333ms
[junit4] 2> 579831 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[A85C3D2AC3D0CB06]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A85C3D2AC3D0CB06-001/solr-instance-001/collection1/data, hostContext=/solr, hostPort=48638}
[junit4] 2> 579832 ERROR (TEST-TestReplicationHandler.testEmptyCommits-seed#[A85C3D2AC3D0CB06]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 579832 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[A85C3D2AC3D0CB06]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
[junit4] 2> 579832 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[A85C3D2AC3D0CB06]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 579832 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[A85C3D2AC3D0CB06]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 579832 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[A85C3D2AC3D0CB06]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-09-06T02:42:32.814Z
[junit4] 2> 579832 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[A85C3D2AC3D0CB06]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_A85C3D2AC3D0CB06-001/solr-instance-001/solr.xml
[junit4] 2> 580010 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[A85C3D2AC3D0CB06]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@5b67311a, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 580065 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[A85C3D2AC3D0CB06]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5b67311a
[junit4] 2> 580073 INFO (TEST-TestReplicationHandler.testEmptyCommits-seed#[A85C3D2AC3D0CB06]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5b67311a
[junit4] 2> 580073 INFO (TEST-TestReplicationHandler.t
[...truncated too long message...]
cher(SolrCore.java:2075)
[junit4] 2> at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:2195)
[junit4] 2> at org.apache.solr.core.SolrCore.initSearcher(SolrCore.java:1071)
[junit4] 2> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:960)
[junit4] 2> ... 31 more
[junit4] 2> Caused by: org.apache.lucene.store.LockObtainFailedException: Lock held by this virtual machine: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrClient_2FF29CB3A774AC20-001/instance-0-001/collection1/data/index/write.lock
[junit4] 2> at org.apache.lucene.store.NativeFSLockFactory.obtainFSLock(NativeFSLockFactory.java:127)
[junit4] 2> at org.apache.lucene.store.FSLockFactory.obtainLock(FSLockFactory.java:41)
[junit4] 2> at org.apache.lucene.store.BaseDirectory.obtainLock(BaseDirectory.java:45)
[junit4] 2> at org.apache.lucene.store.FilterDirectory.obtainLock(FilterDirectory.java:104)
[junit4] 2> at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:947)
[junit4] 2> at org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:119)
[junit4] 2> at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:94)
[junit4] 2> at org.apache.solr.update.DefaultSolrCoreState.createMainIndexWriter(DefaultSolrCoreState.java:257)
[junit4] 2> at org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:131)
[junit4] 2> at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:2036)
[junit4] 2> ... 34 more
[junit4] 2>
[junit4] 2> 157091 INFO (qtp882397021-994) [ ] o.a.s.c.S.Request [collection1] webapp=/solr path=/select params={q=*:*&distrib=false&sort=_docid_+asc&rows=0&wt=javabin&version=2} hits=10 status=0 QTime=0
[junit4] 2> 158215 INFO (qtp882397021-995) [ ] o.a.s.c.S.Request [collection1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=10 status=0 QTime=0
[junit4] 2> 158222 INFO (TEST-TestLBHttpSolrClient.testTwoServers-seed#[2FF29CB3A774AC20]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@45d79ef4{HTTP/1.1,[http/1.1]}{127.0.0.1:47135}
[junit4] 2> 158222 INFO (TEST-TestLBHttpSolrClient.testTwoServers-seed#[2FF29CB3A774AC20]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=668345273
[junit4] 2> 158226 INFO (TEST-TestLBHttpSolrClient.testTwoServers-seed#[2FF29CB3A774AC20]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 158226 INFO (TEST-TestLBHttpSolrClient.testTwoServers-seed#[2FF29CB3A774AC20]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@41e979ee: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@30669710
[junit4] 2> 158234 INFO (TEST-TestLBHttpSolrClient.testTwoServers-seed#[2FF29CB3A774AC20]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 158235 INFO (TEST-TestLBHttpSolrClient.testTwoServers-seed#[2FF29CB3A774AC20]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7970b2c8: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@26377e4d
[junit4] 2> 158238 INFO (TEST-TestLBHttpSolrClient.testTwoServers-seed#[2FF29CB3A774AC20]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 158238 INFO (TEST-TestLBHttpSolrClient.testTwoServers-seed#[2FF29CB3A774AC20]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7e304187: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@73ee7554
[junit4] 2> 158257 INFO (coreCloseExecutor-428-thread-1) [ ] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@642f71ba
[junit4] 2> 158257 INFO (coreCloseExecutor-428-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=1680830906
[junit4] 2> 158257 INFO (coreCloseExecutor-428-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6379e47c: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@3745e1ec
[junit4] 2> 158478 INFO (TEST-TestLBHttpSolrClient.testTwoServers-seed#[2FF29CB3A774AC20]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@76678bf6{/solr,null,UNAVAILABLE}
[junit4] 2> 158483 INFO (TEST-TestLBHttpSolrClient.testTwoServers-seed#[2FF29CB3A774AC20]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@55b94a12{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 158483 INFO (TEST-TestLBHttpSolrClient.testTwoServers-seed#[2FF29CB3A774AC20]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=336269962
[junit4] 2> 158483 INFO (TEST-TestLBHttpSolrClient.testTwoServers-seed#[2FF29CB3A774AC20]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 158483 INFO (TEST-TestLBHttpSolrClient.testTwoServers-seed#[2FF29CB3A774AC20]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@30792c6d: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@fd90c03
[junit4] 2> 158484 INFO (TEST-TestLBHttpSolrClient.testTwoServers-seed#[2FF29CB3A774AC20]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 158484 INFO (TEST-TestLBHttpSolrClient.testTwoServers-seed#[2FF29CB3A774AC20]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5e6db105: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@26377e4d
[junit4] 2> 158485 INFO (TEST-TestLBHttpSolrClient.testTwoServers-seed#[2FF29CB3A774AC20]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 158485 INFO (TEST-TestLBHttpSolrClient.testTwoServers-seed#[2FF29CB3A774AC20]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@de83769: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@73ee7554
[junit4] 2> 159978 INFO (TEST-TestLBHttpSolrClient.testTwoServers-seed#[2FF29CB3A774AC20]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 159990 INFO (coreCloseExecutor-430-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@5dfcda8c
[junit4] 2> 159990 INFO (coreCloseExecutor-430-thread-1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=1576852108
[junit4] 2> 159990 INFO (coreCloseExecutor-430-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7ebdb7a0: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@4ca39229
[junit4] 2> 160018 INFO (TEST-TestLBHttpSolrClient.testTwoServers-seed#[2FF29CB3A774AC20]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@784b594e{/solr,null,UNAVAILABLE}
[junit4] 2> 160020 INFO (TEST-TestLBHttpSolrClient.testTwoServers-seed#[2FF29CB3A774AC20]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testTwoServers
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrClient_2FF29CB3A774AC20-001
[junit4] 2> Sep 06, 2017 4:11:49 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {name=Lucene50(blocksize=128), id=Lucene50(blocksize=128)}, docValues:{}, maxPointsInLeafNode=1090, maxMBSortInHeap=6.232605604607451, sim=RandomSimilarity(queryNorm=false): {}, locale=be, timezone=HST
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=2,free=118925088,total=358612992
[junit4] 2> NOTE: All tests run in this JVM: [SolrZkClientTest, RoundEvaluatorTest, LargeVolumeBinaryJettyTest, TestSpellCheckResponse, JsonValidatorTest, StreamExpressionToExpessionTest, ConcatOperationTest, SolrPingTest, TestRetryUtil, JDBCStreamTest, ModuloEvaluatorTest, TestV2Request, CommonAdminParamsTest, ContentStreamTest, ConcurrentUpdateSolrClientTest, HyperbolicCosineEvaluatorTest, TestDelegationTokenResponse, FloorEvaluatorTest, TestNamedListCodec, AbsoluteValueEvaluatorTest, CumulativeProbabilityEvaluatorTest, UniformDistributionEvaluatorTest, LargeVolumeEmbeddedTest, SolrExampleXMLTest, LargeVolumeJettyTest, MergeIndexesEmbeddedTest, ModifiableSolrParamsTest, TestXMLEscaping, DocumentAnalysisResponseTest, SolrExceptionTest, TestUpdateRequestCodec, FieldAnalysisResponseTest, AnlysisResponseBaseTest, CollectionAdminRequestRequiredParamsTest, TestLBHttpSolrClient]
[junit4] Completed [88/145 (1!)] on J0 in 25.17s, 3 tests, 1 failure <<< FAILURES!
[...truncated 45470 lines...]
[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 37 - Still
unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/37/
3 tests failed.
FAILED: org.apache.solr.update.TestInPlaceUpdatesDistrib.test
Error Message:
inplace_updatable_float didn't match for replica at client: https://127.0.0.1:56342/collection1 expected:<113.0> but was:<1.0>
Stack Trace:
java.lang.AssertionError: inplace_updatable_float didn't match for replica at client: https://127.0.0.1:56342/collection1 expected:<113.0> but was:<1.0>
at __randomizedtesting.SeedInfo.seed([43C8C8997C14F2F9:CB9CF743D2E89F01]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.apache.solr.update.TestInPlaceUpdatesDistrib.assertReplicaValue(TestInPlaceUpdatesDistrib.java:980)
at org.apache.solr.update.TestInPlaceUpdatesDistrib.outOfOrderUpdatesIndividualReplicaTest(TestInPlaceUpdatesDistrib.java:643)
at org.apache.solr.update.TestInPlaceUpdatesDistrib.test(TestInPlaceUpdatesDistrib.java:144)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.client.solrj.TestLBHttpSolrClient.testSimple
Error Message:
expected:<3> but was:<2>
Stack Trace:
java.lang.AssertionError: expected:<3> but was:<2>
at __randomizedtesting.SeedInfo.seed([2A2E22E91EFB491C:129D061739089DCD]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.client.solrj.TestLBHttpSolrClient.testSimple(TestLBHttpSolrClient.java:170)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test
Error Message:
expected:<2> but was:<3>
Stack Trace:
java.lang.AssertionError: expected:<2> but was:<3>
at __randomizedtesting.SeedInfo.seed([43C8C8997C14F2F9:CB9CF743D2E89F01]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.assertSliceAndReplicaCount(SharedFSAutoReplicaFailoverTest.java:403)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.testBasics(SharedFSAutoReplicaFailoverTest.java:248)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test(SharedFSAutoReplicaFailoverTest.java:136)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12445 lines...]
[junit4] Suite: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest
[junit4] 2> 1889663 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[43C8C8997C14F2F9]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_43C8C8997C14F2F9-001/init-core-data-001
[junit4] 2> 1889663 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[43C8C8997C14F2F9]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
[junit4] 2> 1889669 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[43C8C8997C14F2F9]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 1889670 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[43C8C8997C14F2F9]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=None)
[junit4] 2> 1889671 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[43C8C8997C14F2F9]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /d/cl
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 1889777 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[43C8C8997C14F2F9]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 1889809 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[43C8C8997C14F2F9]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 1889819 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[43C8C8997C14F2F9]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 1889851 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[43C8C8997C14F2F9]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_40508_hdfs____bzc0q3/webapp
[junit4] 2> 1890647 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[43C8C8997C14F2F9]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40508
[junit4] 2> 1890804 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[43C8C8997C14F2F9]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 1890805 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[43C8C8997C14F2F9]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 1890813 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[43C8C8997C14F2F9]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_59434_datanode____ermguw/webapp
[junit4] 2> 1891440 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[43C8C8997C14F2F9]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:59434
[junit4] 2> 1891538 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[43C8C8997C14F2F9]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 1891551 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[43C8C8997C14F2F9]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 1891591 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[43C8C8997C14F2F9]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_36718_datanode____.3l4wfy/webapp
[junit4] 2> 1891783 ERROR (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_43C8C8997C14F2F9-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_43C8C8997C14F2F9-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to localhost/127.0.0.1:40666) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 1891823 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x110698dc8a464f: from storage DS-e4830dce-faf4-4169-b81f-d03555031855 node DatanodeRegistration(127.0.0.1:41243, datanodeUuid=2e80fcfd-b1be-43e0-8d2b-fe28927449df, infoPort=40967, infoSecurePort=0, ipcPort=35810, storageInfo=lv=-56;cid=testClusterID;nsid=36264635;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 1891823 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x110698dc8a464f: from storage DS-f08b7515-5904-4e44-91ea-c7052f7bda13 node DatanodeRegistration(127.0.0.1:41243, datanodeUuid=2e80fcfd-b1be-43e0-8d2b-fe28927449df, infoPort=40967, infoSecurePort=0, ipcPort=35810, storageInfo=lv=-56;cid=testClusterID;nsid=36264635;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 1892409 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[43C8C8997C14F2F9]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36718
[junit4] 2> 1892722 ERROR (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_43C8C8997C14F2F9-001/tempDir-001/hdfsBaseDir/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_43C8C8997C14F2F9-001/tempDir-001/hdfsBaseDir/data/data4/]] heartbeating to localhost/127.0.0.1:40666) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 1892747 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x11069913a22681: from storage DS-d1ec3837-e272-479f-99a0-094ba610e596 node DatanodeRegistration(127.0.0.1:60237, datanodeUuid=c6c18b26-c0dd-4c88-84a9-cb0730cb3b95, infoPort=38211, infoSecurePort=0, ipcPort=43652, storageInfo=lv=-56;cid=testClusterID;nsid=36264635;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 1892748 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x11069913a22681: from storage DS-ecac2ba5-5a4c-451f-8899-8fd2f2c130d0 node DatanodeRegistration(127.0.0.1:60237, datanodeUuid=c6c18b26-c0dd-4c88-84a9-cb0730cb3b95, infoPort=38211, infoSecurePort=0, ipcPort=43652, storageInfo=lv=-56;cid=testClusterID;nsid=36264635;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 1892823 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1892830 INFO (Thread-15804) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1892830 INFO (Thread-15804) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1892845 ERROR (Thread-15804) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1893081 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.c.ZkTestServer start zk server on port:59245
[junit4] 2> 1893106 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 1893107 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 1893108 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1893108 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 1893109 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 1893109 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 1893110 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 1893111 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 1893111 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1893112 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 1893113 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 1893113 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
[junit4] 2> 1893203 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 1893203 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2c726882{/d/cl,null,AVAILABLE}
[junit4] 2> 1893204 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@350a1fec{HTTP/1.1,[http/1.1]}{127.0.0.1:52293}
[junit4] 2> 1893204 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.e.j.s.Server Started @1898924ms
[junit4] 2> 1893204 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {replicaType=NRT, hostContext=/d/cl, hostPort=52293, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_43C8C8997C14F2F9-001/control-001/cores}
[junit4] 2> 1893204 ERROR (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1893204 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
[junit4] 2> 1893204 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1893204 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 1893204 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-09-04T15:53:21.829Z
[junit4] 2> 1893213 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1893213 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_43C8C8997C14F2F9-001/control-001/solr.xml
[junit4] 2> 1893217 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@5468e3ad, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 1893219 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:59245/solr
[junit4] 2> 1893259 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [n:127.0.0.1:52293_d%2Fcl ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1893260 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [n:127.0.0.1:52293_d%2Fcl ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:52293_d%2Fcl
[junit4] 2> 1893260 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [n:127.0.0.1:52293_d%2Fcl ] o.a.s.c.Overseer Overseer (id=98601559750672388-127.0.0.1:52293_d%2Fcl-n_0000000000) starting
[junit4] 2> 1893272 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [n:127.0.0.1:52293_d%2Fcl ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52293_d%2Fcl
[junit4] 2> 1893273 INFO (zkCallback-1244-thread-1-processing-n:127.0.0.1:52293_d%2Fcl) [n:127.0.0.1:52293_d%2Fcl ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1893324 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [n:127.0.0.1:52293_d%2Fcl ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5468e3ad
[junit4] 2> 1893333 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [n:127.0.0.1:52293_d%2Fcl ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5468e3ad
[junit4] 2> 1893333 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [n:127.0.0.1:52293_d%2Fcl ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5468e3ad
[junit4] 2> 1893334 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [n:127.0.0.1:52293_d%2Fcl ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_43C8C8997C14F2F9-001/control-001/cores
[junit4] 2> 1893351 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1893351 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:59245/solr ready
[junit4] 2> 1893359 INFO (qtp411990162-22895) [n:127.0.0.1:52293_d%2Fcl ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:52293_d%252Fcl&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1893361 INFO (OverseerThreadFactory-2562-thread-1-processing-n:127.0.0.1:52293_d%2Fcl) [n:127.0.0.1:52293_d%2Fcl ] o.a.s.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 1893522 INFO (qtp411990162-22893) [n:127.0.0.1:52293_d%2Fcl ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1893523 INFO (qtp411990162-22893) [n:127.0.0.1:52293_d%2Fcl ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 1893630 INFO (zkCallback-1244-thread-1-processing-n:127.0.0.1:52293_d%2Fcl) [n:127.0.0.1:52293_d%2Fcl ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1894555 INFO (qtp411990162-22893) [n:127.0.0.1:52293_d%2Fcl c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 1894566 INFO (qtp411990162-22893) [n:127.0.0.1:52293_d%2Fcl c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
[junit4] 2> 1894665 INFO (qtp411990162-22893) [n:127.0.0.1:52293_d%2Fcl c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 1894683 INFO (qtp411990162-22893) [n:127.0.0.1:52293_d%2Fcl c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
[junit4] 2> 1894683 INFO (qtp411990162-22893) [n:127.0.0.1:52293_d%2Fcl c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5468e3ad
[junit4] 2> 1894684 INFO (qtp411990162-22893) [n:127.0.0.1:52293_d%2Fcl c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:40666/solr_hdfs_home
[junit4] 2> 1894684 INFO (qtp411990162-22893) [n:127.0.0.1:52293_d%2Fcl c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 1894684 INFO (qtp411990162-22893) [n:127.0.0.1:52293_d%2Fcl c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1894684 INFO (qtp411990162-22893) [n:127.0.0.1:52293_d%2Fcl c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_43C8C8997C14F2F9-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[hdfs://localhost:40666/solr_hdfs_home/control_collection/core_node2/data/]
[junit4] 2> 1894685 INFO (qtp411990162-22893) [n:127.0.0.1:52293_d%2Fcl c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:40666/solr_hdfs_home/control_collection/core_node2/data/snapshot_metadata
[junit4] 2> 1894691 INFO (qtp411990162-22893) [n:127.0.0.1:52293_d%2Fcl c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1894692 INFO (qtp411990162-22893) [n:127.0.0.1:52293_d%2Fcl c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
[junit4] 2> 1894702 INFO (qtp411990162-22893) [n:127.0.0.1:52293_d%2Fcl c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1894703 INFO (qtp411990162-22893) [n:127.0.0.1:52293_d%2Fcl c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:40666/solr_hdfs_home/control_collection/core_node2/data
[junit4] 2> 1894742 INFO (qtp411990162-22893) [n:127.0.0.1:52293_d%2Fcl c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:40666/solr_hdfs_home/control_collection/core_node2/data/index
[junit4] 2> 1894747 INFO (qtp411990162-22893) [n:127.0.0.1:52293_d%2Fcl c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1894747 INFO (qtp411990162-22893) [n:127.0.0.1:52293_d%2Fcl c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
[junit4] 2> 1894750 INFO (qtp411990162-22893) [n:127.0.0.1:52293_d%2Fcl c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1894750 INFO (qtp411990162-22893) [n:127.0.0.1:52293_d%2Fcl c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1232413146, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 1894774 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:60237 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-f08b7515-5904-4e44-91ea-c7052f7bda13:NORMAL:127.0.0.1:41243|RBW], ReplicaUC[[DISK]DS-ecac2ba5-5a4c-451f-8899-8fd2f2c130d0:NORMAL:127.0.0.1:60237|FINALIZED]]} size 0
[junit4] 2> 1894775 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:41243 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-ecac2ba5-5a4c-451f-8899-8fd2f2c130d0:NORMAL:127.0.0.1:60237|FINALIZED], ReplicaUC[[DISK]DS-e4830dce-faf4-4169-b81f-d03555031855:NORMAL:127.0.0.1:41243|FINALIZED]]} size 0
[junit4] 2> 1894782 WARN (qtp411990162-22893) [n:127.0.0.1:52293_d%2Fcl c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1894855 INFO (qtp411990162-22893) [n:127.0.0.1:52293_d%2Fcl c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 1894855 INFO (qtp411990162-22893) [n:127.0.0.1:52293_d%2Fcl c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1894855 INFO (qtp411990162-22893) [n:127.0.0.1:52293_d%2Fcl c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 1894863 INFO (qtp411990162-22893) [n:127.0.0.1:52293_d%2Fcl c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1894863 INFO (qtp411990162-22893) [n:127.0.0.1:52293_d%2Fcl c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1894865 INFO (qtp411990162-22893) [n:127.0.0.1:52293_d%2Fcl c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@b0f60d2
[junit4] 2> 1894873 INFO (qtp411990162-22893) [n:127.0.0.1:52293_d%2Fcl c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1949f14c[control_collection_shard1_replica_n1] main]
[junit4] 2> 1894874 INFO (qtp411990162-22893) [n:127.0.0.1:52293_d%2Fcl c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1894874 INFO (qtp411990162-22893) [n:127.0.0.1:52293_d%2Fcl c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1894875 INFO (qtp411990162-22893) [n:127.0.0.1:52293_d%2Fcl c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1894876 INFO (searcherExecutor-2565-thread-1-processing-n:127.0.0.1:52293_d%2Fcl x:control_collection_shard1_replica_n1 s:shard1 c:control_collection) [n:127.0.0.1:52293_d%2Fcl c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@1949f14c[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1894876 INFO (qtp411990162-22893) [n:127.0.0.1:52293_d%2Fcl c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577624958141464576
[junit4] 2> 1894881 INFO (qtp411990162-22893) [n:127.0.0.1:52293_d%2Fcl c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1894881 INFO (qtp411990162-22893) [n:127.0.0.1:52293_d%2Fcl c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1894881 INFO (qtp411990162-22893) [n:127.0.0.1:52293_d%2Fcl c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:52293/d/cl/control_collection_shard1_replica_n1/
[junit4] 2> 1894881 INFO (qtp411990162-22893) [n:127.0.0.1:52293_d%2Fcl c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1894881 INFO (qtp411990162-22893) [n:127.0.0.1:52293_d%2Fcl c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:52293/d/cl/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 1894881 INFO (qtp411990162-22893) [n:127.0.0.1:52293_d%2Fcl c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 1894884 INFO (qtp411990162-22893) [n:127.0.0.1:52293_d%2Fcl c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:52293/d/cl/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 1895031 INFO (zkCallback-1244-thread-1-processing-n:127.0.0.1:52293_d%2Fcl) [n:127.0.0.1:52293_d%2Fcl ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1895032 INFO (qtp411990162-22893) [n:127.0.0.1:52293_d%2Fcl c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1895034 INFO (qtp411990162-22893) [n:127.0.0.1:52293_d%2Fcl c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1511
[junit4] 2> 1895041 INFO (qtp411990162-22895) [n:127.0.0.1:52293_d%2Fcl ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 1895134 INFO (zkCallback-1244-thread-1-processing-n:127.0.0.1:52293_d%2Fcl) [n:127.0.0.1:52293_d%2Fcl ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1895368 INFO (OverseerCollectionConfigSetProcessor-98601559750672388-127.0.0.1:52293_d%2Fcl-n_0000000000) [n:127.0.0.1:52293_d%2Fcl ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1896041 INFO (qtp411990162-22895) [n:127.0.0.1:52293_d%2Fcl ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:52293_d%252Fcl&wt=javabin&version=2} status=0 QTime=2682
[junit4] 2> 1896049 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1896050 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:59245/solr ready
[junit4] 2> 1896050 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 1896050 INFO (qtp411990162-22896) [n:127.0.0.1:52293_d%2Fcl ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1896056 INFO (OverseerThreadFactory-2562-thread-2-processing-n:127.0.0.1:52293_d%2Fcl) [n:127.0.0.1:52293_d%2Fcl ] o.a.s.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 1896057 WARN (OverseerThreadFactory-2562-thread-2-processing-n:127.0.0.1:52293_d%2Fcl) [n:127.0.0.1:52293_d%2Fcl ] o.a.s.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
[junit4] 2> 1896261 INFO (qtp411990162-22896) [n:127.0.0.1:52293_d%2Fcl ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 1896261 INFO (qtp411990162-22896) [n:127.0.0.1:52293_d%2Fcl ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=1&wt=javabin&version=2} status=0 QTime=210
[junit4] 2> 1896359 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_43C8C8997C14F2F9-001/shard-1-001 of type NRT
[junit4] 2> 1896360 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 1896361 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5fd2192f{/d/cl,null,AVAILABLE}
[junit4] 2> 1896361 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@4ca9680{HTTP/1.1,[http/1.1]}{127.0.0.1:50370}
[junit4] 2> 1896361 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.e.j.s.Server Started @1902081ms
[junit4] 2> 1896361 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml, hostContext=/d/cl, hostPort=50370, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_43C8C8997C14F2F9-001/shard-1-001/cores}
[junit4] 2> 1896361 ERROR (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1896361 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
[junit4] 2> 1896361 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1896361 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 1896362 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-09-04T15:53:24.986Z
[junit4] 2> 1896367 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1896367 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_43C8C8997C14F2F9-001/shard-1-001/solr.xml
[junit4] 2> 1896371 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@5468e3ad, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 1896374 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:59245/solr
[junit4] 2> 1896379 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [n:127.0.0.1:50370_d%2Fcl ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1896380 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [n:127.0.0.1:50370_d%2Fcl ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1896382 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [n:127.0.0.1:50370_d%2Fcl ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50370_d%2Fcl
[junit4] 2> 1896382 INFO (zkCallback-1244-thread-1-processing-n:127.0.0.1:52293_d%2Fcl) [n:127.0.0.1:52293_d%2Fcl ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1896395 INFO (zkCallback-1251-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1896406 INFO (zkCallback-1256-thread-2-processing-n:127.0.0.1:50370_d%2Fcl) [n:127.0.0.1:50370_d%2Fcl ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1896465 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [n:127.0.0.1:50370_d%2Fcl ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5468e3ad
[junit4] 2> 1896475 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [n:127.0.0.1:50370_d%2Fcl ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5468e3ad
[junit4] 2> 1896475 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [n:127.0.0.1:50370_d%2Fcl ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5468e3ad
[junit4] 2> 1896476 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [n:127.0.0.1:50370_d%2Fcl ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_43C8C8997C14F2F9-001/shard-1-001/cores
[junit4] 2> 1896503 INFO (qtp411990162-22892) [n:127.0.0.1:52293_d%2Fcl ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:50370_d%252Fcl&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1896508 INFO (OverseerCollectionConfigSetProcessor-98601559750672388-127.0.0.1:52293_d%2Fcl-n_0000000000) [n:127.0.0.1:52293_d%2Fcl ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1896508 INFO (OverseerThreadFactory-2562-thread-3-processing-n:127.0.0.1:52293_d%2Fcl) [n:127.0.0.1:52293_d%2Fcl ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:50370_d%2Fcl for creating new replica
[junit4] 2> 1896511 INFO (qtp377225147-22946) [n:127.0.0.1:50370_d%2Fcl ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1896511 INFO (qtp377225147-22946) [n:127.0.0.1:50370_d%2Fcl ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 1897529 INFO (qtp377225147-22946) [n:127.0.0.1:50370_d%2Fcl c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 1897540 INFO (qtp377225147-22946) [n:127.0.0.1:50370_d%2Fcl c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.s.IndexSchema [collection1_shard2_replica_n41] Schema name=test
[junit4] 2> 1897630 INFO (qtp377225147-22946) [n:127.0.0.1:50370_d%2Fcl c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 1897645 INFO (qtp377225147-22946) [n:127.0.0.1:50370_d%2Fcl c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n41' using configuration from collection collection1, trusted=true
[junit4] 2> 1897646 INFO (qtp377225147-22946) [n:127.0.0.1:50370_d%2Fcl c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_n41' (registry 'solr.core.collection1.shard2.replica_n41') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5468e3ad
[junit4] 2> 1897646 INFO (qtp377225147-22946) [n:127.0.0.1:50370_d%2Fcl c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:40666/solr_hdfs_home
[junit4] 2> 1897646 INFO (qtp377225147-22946) [n:127.0.0.1:50370_d%2Fcl c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 1897646 INFO (qtp377225147-22946) [n:127.0.0.1:50370_d%2Fcl c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1897646 INFO (qtp377225147-22946) [n:127.0.0.1:50370_d%2Fcl c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.SolrCore [[collection1_shard2_replica_n41] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_43C8C8997C14F2F9-001/shard-1-001/cores/collection1_shard2_replica_n41], dataDir=[hdfs://localhost:40666/solr_hdfs_home/collection1/core_node42/data/]
[junit4] 2> 1897647 INFO (qtp377225147-22946) [n:127.0.0.1:50370_d%2Fcl c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:40666/solr_hdfs_home/collection1/core_node42/data/snapshot_metadata
[junit4] 2> 1897657 INFO (qtp377225147-22946) [n:127.0.0.1:50370_d%2Fcl c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1897657 INFO (qtp377225147-22946) [n:127.0.0.1:50370_d%2Fcl c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
[junit4] 2> 1897667 INFO (qtp377225147-22946) [n:127.0.0.1:50370_d%2Fcl c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1897668 INFO (qtp377225147-22946) [n:127.0.0.1:50370_d%2Fcl c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:40666/solr_hdfs_home/collection1/core_node42/data
[junit4] 2> 1897680 INFO (qtp377225147-22946) [n:127.0.0.1:50370_d%2Fcl c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:40666/solr_hdfs_home/collection1/core_node42/data/index
[junit4] 2> 1897685 INFO (qtp377225147-22946) [n:127.0.0.1:50370_d%2Fcl c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1897685 INFO (qtp377225147-22946) [n:127.0.0.1:50370_d%2Fcl c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
[junit4] 2> 1897696 INFO (qtp377225147-22946) [n:127.0.0.1:50370_d%2Fcl c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1897697 INFO (qtp377225147-22946) [n:127.0.0.1:50370_d%2Fcl c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1232413146, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 1897714 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:41243 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-d1ec3837-e272-479f-99a0-094ba610e596:NORMAL:127.0.0.1:60237|RBW], ReplicaUC[[DISK]DS-f08b7515-5904-4e44-91ea-c7052f7bda13:NORMAL:127.0.0.1:41243|RBW]]} size 0
[junit4] 2> 1897717 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:60237 is added to blk_1073741826_1002 size 72
[junit4] 2> 1897719 WARN (qtp377225147-22946) [n:127.0.0.1:50370_d%2Fcl c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1897852 INFO (qtp377225147-22946) [n:127.0.0.1:50370_d%2Fcl c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 1897852 INFO (qtp377225147-22946) [n:127.0.0.1:50370_d%2Fcl c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1897852 INFO (qtp377225147-22946) [n:127.0.0.1:50370_d%2Fcl c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 1897866 INFO (qtp377225147-22946) [n:127.0.0.1:50370_d%2Fcl c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1897866 INFO (qtp377225147-22946) [n:127.0.0.1:50370_d%2Fcl c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1897867 INFO (qtp377225147-22946) [n:127.0.0.1:50370_d%2Fcl c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@353d7eff
[junit4] 2> 1897870 INFO (qtp377225147-22946) [n:127.0.0.1:50370_d%2Fcl c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.s.SolrIndexSearcher Opening [Searcher@5c43de07[collection1_shard2_replica_n41] main]
[junit4] 2> 1897872 INFO (qtp377225147-22946) [n:127.0.0.1:50370_d%2Fcl c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1897872 INFO (qtp377225147-22946) [n:127.0.0.1:50370_d%2Fcl c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1897872 INFO (qtp377225147-22946) [n:127.0.0.1:50370_d%2Fcl c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1897873 INFO (qtp377225147-22946) [n:127.0.0.1:50370_d%2Fcl c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577624961284046848
[junit4] 2> 1897875 INFO (searcherExecutor-2576-thread-1-processing-n:127.0.0.1:50370_d%2Fcl x:collection1_shard2_replica_n41 s:shard2 c:collection1) [n:127.0.0.1:50370_d%2Fcl c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.SolrCore [collection1_shard2_replica_n41] Registered new searcher Searcher@5c43de07[collection1_shard2_replica_n41] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1897879 INFO (qtp377225147-22946) [n:127.0.0.1:50370_d%2Fcl c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1897879 INFO (qtp377225147-22946) [n:127.0.0.1:50370_d%2Fcl c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1897879 INFO (qtp377225147-22946) [n:127.0.0.1:50370_d%2Fcl c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:50370/d/cl/collection1_shard2_replica_n41/
[junit4] 2> 1897879 INFO (qtp377225147-22946) [n:127.0.0.1:50370_d%2Fcl c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1897879 INFO (qtp377225147-22946) [n:127.0.0.1:50370_d%2Fcl c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.SyncStrategy http://127.0.0.1:50370/d/cl/collection1_shard2_replica_n41/ has no replicas
[junit4] 2> 1897879 INFO (qtp377225147-22946) [n:127.0.0.1:50370_d%2Fcl c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 1897882 INFO (qtp377225147-22946) [n:127.0.0.1:50370_d%2Fcl c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:50370/d/cl/collection1_shard2_replica_n41/ shard2
[junit4] 2> 1897986 INFO (qtp377225147-22946) [n:127.0.0.1:50370_d%2Fcl c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1897988 INFO (qtp377225147-22946) [n:127.0.0.1:50370_d%2Fcl c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1476
[junit4] 2> 1897989 INFO (qtp411990162-22892) [n:127.0.0.1:52293_d%2Fcl ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:50370_d%252Fcl&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2} status=0 QTime=1485
[junit4] 2> 1898115 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_43C8C8997C14F2F9-001/shard-2-001 of type NRT
[junit4] 2> 1898116 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 1898117 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@63e9fff9{/d/cl,null,AVAILABLE}
[junit4] 2> 1898117 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@6b2bc584{HTTP/1.1,[http/1.1]}{127.0.0.1:43106}
[junit4] 2> 1898117 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.e.j.s.Server Started @1903837ms
[junit4] 2> 1898117 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/d/cl, hostPort=43106, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_43C8C8997C14F2F9-001/shard-2-001/cores}
[junit4] 2> 1898117 ERROR (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1898126 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
[junit4] 2> 1898126 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1898126 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 1898126 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-09-04T15:53:26.751Z
[junit4] 2> 1898137 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1898137 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_43C8C8997C14F2F9-001/shard-2-001/solr.xml
[junit4] 2> 1898143 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@5468e3ad, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 1898145 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:59245/solr
[junit4] 2> 1898154 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [n:127.0.0.1:43106_d%2Fcl ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1898155 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [n:127.0.0.1:43106_d%2Fcl ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1898156 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [n:127.0.0.1:43106_d%2Fcl ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43106_d%2Fcl
[junit4] 2> 1898157 INFO (zkCallback-1251-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1898166 INFO (zkCallback-1244-thread-1-processing-n:127.0.0.1:52293_d%2Fcl) [n:127.0.0.1:52293_d%2Fcl ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1898166 INFO (zkCallback-1256-thread-2-processing-n:127.0.0.1:50370_d%2Fcl) [n:127.0.0.1:50370_d%2Fcl ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1898198 INFO (zkCallback-1262-thread-2-processing-n:127.0.0.1:43106_d%2Fcl) [n:127.0.0.1:43106_d%2Fcl ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1898270 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [n:127.0.0.1:43106_d%2Fcl ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5468e3ad
[junit4] 2> 1898314 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [n:127.0.0.1:43106_d%2Fcl ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5468e3ad
[junit4] 2> 1898314 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [n:127.0.0.1:43106_d%2Fcl ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5468e3ad
[junit4] 2> 1898315 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [n:127.0.0.1:43106_d%2Fcl ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_43C8C8997C14F2F9-001/shard-2-001/cores
[junit4] 2> 1898354 INFO (qtp1928808999-22983) [n:127.0.0.1:43106_d%2Fcl ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:43106_d%252Fcl&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1898356 INFO (OverseerCollectionConfigSetProcessor-98601559750672388-127.0.0.1:52293_d%2Fcl-n_0000000000) [n:127.0.0.1:52293_d%2Fcl ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1898359 INFO (OverseerThreadFactory-2562-thread-4-processing-n:127.0.0.1:52293_d%2Fcl) [n:127.0.0.1:52293_d%2Fcl ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:43106_d%2Fcl for creating new replica
[junit4] 2> 1898361 INFO (qtp1928808999-22981) [n:127.0.0.1:43106_d%2Fcl ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n43&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1898361 INFO (qtp1928808999-22981) [n:127.0.0.1:43106_d%2Fcl ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 1899376 INFO (qtp1928808999-22981) [n:127.0.0.1:43106_d%2Fcl c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 1899388 INFO (qtp1928808999-22981) [n:127.0.0.1:43106_d%2Fcl c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.s.IndexSchema [collection1_shard1_replica_n43] Schema name=test
[junit4] 2> 1899467 INFO (qtp1928808999-22981) [n:127.0.0.1:43106_d%2Fcl c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 1899483 INFO (qtp1928808999-22981) [n:127.0.0.1:43106_d%2Fcl c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n43' using configuration from collection collection1, trusted=true
[junit4] 2> 1899483 INFO (qtp1928808999-22981) [n:127.0.0.1:43106_d%2Fcl c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n43' (registry 'solr.core.collection1.shard1.replica_n43') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5468e3ad
[junit4] 2> 1899483 INFO (qtp1928808999-22981) [n:127.0.0.1:43106_d%2Fcl c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:40666/solr_hdfs_home
[junit4] 2> 1899483 INFO (qtp1928808999-22981) [n:127.0.0.1:43106_d%2Fcl c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 1899483 INFO (qtp1928808999-22981) [n:127.0.0.1:43106_d%2Fcl c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1899483 INFO (qtp1928808999-22981) [n:127.0.0.1:43106_d%2Fcl c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.SolrCore [[collection1_shard1_replica_n43] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_43C8C8997C14F2F9-001/shard-2-001/cores/collection1_shard1_replica_n43], dataDir=[hdfs://localhost:40666/solr_hdfs_home/collection1/core_node44/data/]
[junit4] 2> 1899485 INFO (qtp1928808999-22981) [n:127.0.0.1:43106_d%2Fcl c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:40666/solr_hdfs_home/collection1/core_node44/data/snapshot_metadata
[junit4] 2> 1899491 INFO (qtp1928808999-22981) [n:127.0.0.1:43106_d%2Fcl c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1899491 INFO (qtp1928808999-22981) [n:127.0.0.1:43106_d%2Fcl c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
[junit4] 2> 1900031 INFO (qtp1928808999-22981) [n:127.0.0.1:43106_d%2Fcl c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1900032 INFO (qtp1928808999-22981) [n:127.0.0.1:43106_d%2Fcl c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:40666/solr_hdfs_home/collection1/core_node44/data
[junit4] 2> 1900049 INFO (qtp1928808999-22981) [n:127.0.0.1:43106_d%2Fcl c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:40666/solr_hdfs_home/collection1/core_node44/data/index
[junit4] 2> 1900055 INFO (qtp1928808999-22981) [n:127.0.0.1:43106_d%2Fcl c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1900055 INFO (qtp1928808999-22981) [n:127.0.0.1:43106_d%2Fcl c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
[junit4] 2> 1900065 INFO (qtp1928808999-22981) [n:127.0.0.1:43106_d%2Fcl c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1900065 INFO (qtp1928808999-22981) [n:127.0.0.1:43106_d%2Fcl c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1232413146, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 1900072 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:41243 is added to blk_1073741827_1003{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-ecac2ba5-5a4c-451f-8899-8fd2f2c130d0:NORMAL:127.0.0.1:60237|RBW], ReplicaUC[[DISK]DS-e4830dce-faf4-4169-b81f-d03555031855:NORMAL:127.0.0.1:41243|FINALIZED]]} size 0
[junit4] 2> 1900073 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:60237 is added to blk_1073741827_1003{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-ecac2ba5-5a4c-451f-8899-8fd2f2c130d0:NORMAL:127.0.0.1:60237|RBW], ReplicaUC[[DISK]DS-e4830dce-faf4-4169-b81f-d03555031855:NORMAL:127.0.0.1:41243|FINALIZED]]} size 0
[junit4] 2> 1900085 WARN (qtp1928808999-22981) [n:127.0.0.1:43106_d%2Fcl c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1900129 INFO (qtp1928808999-22981) [n:127.0.0.1:43106_d%2Fcl c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 1900130 INFO (qtp1928808999-22981) [n:127.0.0.1:43106_d%2Fcl c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1900130 INFO (qtp1928808999-22981) [n:127.0.0.1:43106_d%2Fcl c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 1900139 INFO (qtp1928808999-22981) [n:127.0.0.1:43106_d%2Fcl c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1900139 INFO (qtp1928808999-22981) [n:127.0.0.1:43106_d%2Fcl c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1900140 INFO (qtp1928808999-22981) [n:127.0.0.1:43106_d%2Fcl c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@1865965b
[junit4] 2> 1900145 INFO (qtp1928808999-22981) [n:127.0.0.1:43106_d%2Fcl c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.s.SolrIndexSearcher Opening [Searcher@62b98f6e[collection1_shard1_replica_n43] main]
[junit4] 2> 1900155 INFO (qtp1928808999-22981) [n:127.0.0.1:43106_d%2Fcl c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1900155 INFO (qtp1928808999-22981) [n:127.0.0.1:43106_d%2Fcl c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1900156 INFO (qtp1928808999-22981) [n:127.0.0.1:43106_d%2Fcl c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1900156 INFO (qtp1928808999-22981) [n:127.0.0.1:43106_d%2Fcl c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577624963677945856
[junit4] 2> 1900159 INFO (searcherExecutor-2587-thread-1-processing-n:127.0.0.1:43106_d%2Fcl x:collection1_shard1_replica_n43 s:shard1 c:collection1) [n:127.0.0.1:43106_d%2Fcl c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.SolrCore [collection1_shard1_replica_n43] Registered new searcher Searcher@62b98f6e[collection1_shard1_replica_n43] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1900163 INFO (qtp1928808999-22981) [n:127.0.0.1:43106_d%2Fcl c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1900163 INFO (qtp1928808999-22981) [n:127.0.0.1:43106_d%2Fcl c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1900163 INFO (qtp1928808999-22981) [n:127.0.0.1:43106_d%2Fcl c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:43106/d/cl/collection1_shard1_replica_n43/
[junit4] 2> 1900163 INFO (qtp1928808999-22981) [n:127.0.0.1:43106_d%2Fcl c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1900163 INFO (qtp1928808999-22981) [n:127.0.0.1:43106_d%2Fcl c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.SyncStrategy http://127.0.0.1:43106/d/cl/collection1_shard1_replica_n43/ has no replicas
[junit4] 2> 1900163 INFO (qtp1928808999-22981) [n:127.0.0.1:43106_d%2Fcl c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 1900166 INFO (qtp1928808999-22981) [n:127.0.0.1:43106_d%2Fcl c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:43106/d/cl/collection1_shard1_replica_n43/ shard1
[junit4] 2> 1900270 INFO (qtp1928808999-22981) [n:127.0.0.1:43106_d%2Fcl c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1900273 INFO (qtp1928808999-22981) [n:127.0.0.1:43106_d%2Fcl c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n43&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1911
[junit4] 2> 1900274 INFO (qtp1928808999-22983) [n:127.0.0.1:43106_d%2Fcl ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:43106_d%252Fcl&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=1919
[junit4] 2> 1900357 INFO (OverseerCollectionConfigSetProcessor-98601559750672388-127.0.0.1:52293_d%2Fcl-n_0000000000) [n:127.0.0.1:52293_d%2Fcl ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000006 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1900367 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_43C8C8997C14F2F9-001/shard-3-001 of type NRT
[junit4] 2> 1900368 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 1900369 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@65d26c5{/d/cl,null,AVAILABLE}
[junit4] 2> 1900369 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@5d069d8e{HTTP/1.1,[http/1.1]}{127.0.0.1:50130}
[junit4] 2> 1900369 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.e.j.s.Server Started @1906089ms
[junit4] 2> 1900369 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml, hostContext=/d/cl, hostPort=50130, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_43C8C8997C14F2F9-001/shard-3-001/cores}
[junit4] 2> 1900369 ERROR (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1900380 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
[junit4] 2> 1900381 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1900381 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 1900381 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-09-04T15:53:29.006Z
[junit4] 2> 1900392 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1900392 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_43C8C8997C14F2F9-001/shard-3-001/solr.xml
[junit4] 2> 1900396 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[43C8C8997C14F2F9]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserve
[...truncated too long message...]
{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 61827 INFO (TEST-TestLBHttpSolrClient.testReliability-seed#[2A2E22E91EFB491C]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=2111278771
[junit4] 2> 61827 INFO (TEST-TestLBHttpSolrClient.testReliability-seed#[2A2E22E91EFB491C]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 61827 INFO (TEST-TestLBHttpSolrClient.testReliability-seed#[2A2E22E91EFB491C]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@601f5c46: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@1b0941bd
[junit4] 2> 61831 INFO (TEST-TestLBHttpSolrClient.testReliability-seed#[2A2E22E91EFB491C]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 61831 INFO (TEST-TestLBHttpSolrClient.testReliability-seed#[2A2E22E91EFB491C]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@47f1420f: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@206f1d3d
[junit4] 2> 61834 INFO (TEST-TestLBHttpSolrClient.testReliability-seed#[2A2E22E91EFB491C]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 61834 INFO (TEST-TestLBHttpSolrClient.testReliability-seed#[2A2E22E91EFB491C]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@510e6573: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@2353789b
[junit4] 2> 61840 INFO (TEST-TestLBHttpSolrClient.testReliability-seed#[2A2E22E91EFB491C]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 61907 INFO (coreCloseExecutor-164-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@4354d44a
[junit4] 2> 61907 INFO (coreCloseExecutor-164-thread-1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=1129632842
[junit4] 2> 61907 INFO (coreCloseExecutor-164-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6c8cd963: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@3c534bfb
[junit4] 2> 61945 INFO (TEST-TestLBHttpSolrClient.testReliability-seed#[2A2E22E91EFB491C]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@493531b9{/solr,null,UNAVAILABLE}
[junit4] 2> 61963 INFO (TEST-TestLBHttpSolrClient.testReliability-seed#[2A2E22E91EFB491C]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@72676b44{SSL,[ssl, http/1.1]}{127.0.0.1:57618}
[junit4] 2> 61963 INFO (TEST-TestLBHttpSolrClient.testReliability-seed#[2A2E22E91EFB491C]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=2046599605
[junit4] 2> 61963 INFO (TEST-TestLBHttpSolrClient.testReliability-seed#[2A2E22E91EFB491C]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 61963 INFO (TEST-TestLBHttpSolrClient.testReliability-seed#[2A2E22E91EFB491C]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2b1cdf0f: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@395e4461
[junit4] 2> 61968 INFO (TEST-TestLBHttpSolrClient.testReliability-seed#[2A2E22E91EFB491C]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 61968 INFO (TEST-TestLBHttpSolrClient.testReliability-seed#[2A2E22E91EFB491C]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@639346e6: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@206f1d3d
[junit4] 2> 61980 INFO (TEST-TestLBHttpSolrClient.testReliability-seed#[2A2E22E91EFB491C]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 61980 INFO (TEST-TestLBHttpSolrClient.testReliability-seed#[2A2E22E91EFB491C]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@37ea8564: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@2353789b
[junit4] 2> 61986 INFO (coreCloseExecutor-166-thread-1) [ ] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@7c608a38
[junit4] 2> 61987 INFO (coreCloseExecutor-166-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=2086701624
[junit4] 2> 61987 INFO (coreCloseExecutor-166-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5ee43b54: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@4276e779
[junit4] 2> 62032 INFO (TEST-TestLBHttpSolrClient.testReliability-seed#[2A2E22E91EFB491C]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5bafb0c7{/solr,null,UNAVAILABLE}
[junit4] 2> 62042 INFO (TEST-TestLBHttpSolrClient.testReliability-seed#[2A2E22E91EFB491C]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@51400abc{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 62042 INFO (TEST-TestLBHttpSolrClient.testReliability-seed#[2A2E22E91EFB491C]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=558220687
[junit4] 2> 62044 INFO (TEST-TestLBHttpSolrClient.testReliability-seed#[2A2E22E91EFB491C]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 62044 INFO (TEST-TestLBHttpSolrClient.testReliability-seed#[2A2E22E91EFB491C]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@e516686: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@2ee2f978
[junit4] 2> 62045 INFO (TEST-TestLBHttpSolrClient.testReliability-seed#[2A2E22E91EFB491C]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 62045 INFO (TEST-TestLBHttpSolrClient.testReliability-seed#[2A2E22E91EFB491C]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@75134f5: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@206f1d3d
[junit4] 2> 62046 INFO (TEST-TestLBHttpSolrClient.testReliability-seed#[2A2E22E91EFB491C]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 62046 INFO (TEST-TestLBHttpSolrClient.testReliability-seed#[2A2E22E91EFB491C]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1b9c3aa8: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@2353789b
[junit4] 2> 62046 INFO (TEST-TestLBHttpSolrClient.testReliability-seed#[2A2E22E91EFB491C]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 62058 INFO (coreCloseExecutor-168-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@4c2620a2
[junit4] 2> 62059 INFO (coreCloseExecutor-168-thread-1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=1277567138
[junit4] 2> 62059 INFO (coreCloseExecutor-168-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@29bb85e0: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@227633b0
[junit4] 2> 62126 INFO (TEST-TestLBHttpSolrClient.testReliability-seed#[2A2E22E91EFB491C]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5e975afb{/solr,null,UNAVAILABLE}
[junit4] 2> 62129 INFO (TEST-TestLBHttpSolrClient.testReliability-seed#[2A2E22E91EFB491C]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testReliability
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-solrj/test/J0/temp/solr.client.solrj.TestLBHttpSolrClient_2A2E22E91EFB491C-001
[junit4] 2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarity(queryNorm=false): {}, locale=de-AT, timezone=Canada/Saskatchewan
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=2,free=248903440,total=361758720
[junit4] 2> NOTE: All tests run in this JVM: [BasicHttpSolrClientTest, ContentStreamTest, ConcatOperationTest, CloudSolrClientMultiConstructorTest, SolrZkClientTest, TestRetryUtil, TestXMLEscaping, ArcTangentEvaluatorTest, CommonAdminParamsTest, TestDelegationTokenRequest, EqualToEvaluatorTest, TestLBHttpSolrClient]
[junit4] Completed [22/145 (1!)] on J0 in 22.35s, 3 tests, 1 failure <<< FAILURES!
[...truncated 45644 lines...]
[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 36 - Failure
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/36/
3 tests failed.
FAILED: org.apache.lucene.index.TestBinaryDocValuesUpdates.testTonsOfUpdates
Error Message:
_2_4.fnm in dir=RAMDirectory@28ad733b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1e6fb1f1
Stack Trace:
java.io.FileNotFoundException: _2_4.fnm in dir=RAMDirectory@28ad733b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1e6fb1f1
at __randomizedtesting.SeedInfo.seed([401AC1DFEE4FDDDA:383F1FD40C6FF238]:0)
at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:750)
at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:119)
at org.apache.lucene.store.MockDirectoryWrapper.openChecksumInput(MockDirectoryWrapper.java:1072)
at org.apache.lucene.codecs.lucene60.Lucene60FieldInfosFormat.read(Lucene60FieldInfosFormat.java:113)
at org.apache.lucene.index.SegmentReader.initFieldInfos(SegmentReader.java:190)
at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:93)
at org.apache.lucene.index.ReadersAndUpdates.writeFieldUpdates(ReadersAndUpdates.java:688)
at org.apache.lucene.index.IndexWriter$ReaderPool.writeSomeDocValuesUpdates(IndexWriter.java:703)
at org.apache.lucene.index.FrozenBufferedUpdates.apply(FrozenBufferedUpdates.java:331)
at org.apache.lucene.index.DocumentsWriter$ResolveUpdatesEvent.process(DocumentsWriter.java:723)
at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5060)
at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5048)
at org.apache.lucene.index.IndexWriter.updateDocValues(IndexWriter.java:1879)
at org.apache.lucene.index.TestBinaryDocValuesUpdates.testTonsOfUpdates(TestBinaryDocValuesUpdates.java:1323)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.lucene.index.TestNumericDocValuesUpdates.testTonsOfUpdates
Error Message:
_1_7.fnm in dir=RAMDirectory@5a562f20 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2e6b74dd
Stack Trace:
java.io.FileNotFoundException: _1_7.fnm in dir=RAMDirectory@5a562f20 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2e6b74dd
at __randomizedtesting.SeedInfo.seed([401AC1DFEE4FDDDA:383F1FD40C6FF238]:0)
at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:750)
at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:119)
at org.apache.lucene.store.MockDirectoryWrapper.openChecksumInput(MockDirectoryWrapper.java:1072)
at org.apache.lucene.codecs.lucene60.Lucene60FieldInfosFormat.read(Lucene60FieldInfosFormat.java:113)
at org.apache.lucene.index.SegmentReader.initFieldInfos(SegmentReader.java:190)
at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:93)
at org.apache.lucene.index.ReadersAndUpdates.writeFieldUpdates(ReadersAndUpdates.java:688)
at org.apache.lucene.index.IndexWriter$ReaderPool.writeSomeDocValuesUpdates(IndexWriter.java:703)
at org.apache.lucene.index.FrozenBufferedUpdates.apply(FrozenBufferedUpdates.java:331)
at org.apache.lucene.index.DocumentsWriter$ResolveUpdatesEvent.process(DocumentsWriter.java:723)
at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5060)
at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5048)
at org.apache.lucene.index.IndexWriter.updateDocValues(IndexWriter.java:1879)
at org.apache.lucene.index.TestNumericDocValuesUpdates.testTonsOfUpdates(TestNumericDocValuesUpdates.java:1557)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test
Error Message:
Timeout waiting for all live and active
Stack Trace:
java.lang.AssertionError: Timeout waiting for all live and active
at __randomizedtesting.SeedInfo.seed([9799A1A109A32D92:1FCD9E7BA75F406A]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.testBasics(SharedFSAutoReplicaFailoverTest.java:317)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test(SharedFSAutoReplicaFailoverTest.java:136)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 412 lines...]
[junit4] Suite: org.apache.lucene.index.TestBinaryDocValuesUpdates
[junit4] 1> TEST: isNRT=false reader1=StandardDirectoryReader(segments_1:4 _0(7.1.0):C2)
[junit4] 1> TEST: now reopen
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestBinaryDocValuesUpdates -Dtests.method=testTonsOfUpdates -Dtests.seed=401AC1DFEE4FDDDA -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=hu -Dtests.timezone=Australia/Tasmania -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 49.4s J0 | TestBinaryDocValuesUpdates.testTonsOfUpdates <<<
[junit4] > Throwable #1: java.io.FileNotFoundException: _2_4.fnm in dir=RAMDirectory@28ad733b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1e6fb1f1
[junit4] > at __randomizedtesting.SeedInfo.seed([401AC1DFEE4FDDDA:383F1FD40C6FF238]:0)
[junit4] > at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:750)
[junit4] > at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:119)
[junit4] > at org.apache.lucene.store.MockDirectoryWrapper.openChecksumInput(MockDirectoryWrapper.java:1072)
[junit4] > at org.apache.lucene.codecs.lucene60.Lucene60FieldInfosFormat.read(Lucene60FieldInfosFormat.java:113)
[junit4] > at org.apache.lucene.index.SegmentReader.initFieldInfos(SegmentReader.java:190)
[junit4] > at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:93)
[junit4] > at org.apache.lucene.index.ReadersAndUpdates.writeFieldUpdates(ReadersAndUpdates.java:688)
[junit4] > at org.apache.lucene.index.IndexWriter$ReaderPool.writeSomeDocValuesUpdates(IndexWriter.java:703)
[junit4] > at org.apache.lucene.index.FrozenBufferedUpdates.apply(FrozenBufferedUpdates.java:331)
[junit4] > at org.apache.lucene.index.DocumentsWriter$ResolveUpdatesEvent.process(DocumentsWriter.java:723)
[junit4] > at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5060)
[junit4] > at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5048)
[junit4] > at org.apache.lucene.index.IndexWriter.updateDocValues(IndexWriter.java:1879)
[junit4] > at org.apache.lucene.index.TestBinaryDocValuesUpdates.testTonsOfUpdates(TestBinaryDocValuesUpdates.java:1323)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/build/core/test/J0/temp/lucene.index.TestBinaryDocValuesUpdates_401AC1DFEE4FDDDA-001
[junit4] 2> NOTE: test params are: codec=FastCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST, chunkSize=3, maxDocsPerChunk=7, blockSize=5), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST, chunkSize=3, blockSize=5)), sim=RandomSimilarity(queryNorm=false): {}, locale=hu, timezone=Australia/Tasmania
[junit4] 2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=256163392,total=473956352
[junit4] 2> NOTE: All tests run in this JVM: [TestSynonymQuery, TestBooleanScorer, TestDoubleRangeFieldQueries, TestSimilarity, TestLongBitSet, TestTerm, TestSort, TestDocValuesIndexing, TestIndexReaderClose, TestBinaryDocValuesUpdates]
[junit4] Completed [54/454 (1!)] on J0 in 142.72s, 29 tests, 1 error <<< FAILURES!
[...truncated 303 lines...]
[junit4] Suite: org.apache.lucene.index.TestNumericDocValuesUpdates
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestNumericDocValuesUpdates -Dtests.method=testTonsOfUpdates -Dtests.seed=401AC1DFEE4FDDDA -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=ga -Dtests.timezone=America/Porto_Velho -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 22.4s J1 | TestNumericDocValuesUpdates.testTonsOfUpdates <<<
[junit4] > Throwable #1: java.io.FileNotFoundException: _1_7.fnm in dir=RAMDirectory@5a562f20 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2e6b74dd
[junit4] > at __randomizedtesting.SeedInfo.seed([401AC1DFEE4FDDDA:383F1FD40C6FF238]:0)
[junit4] > at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:750)
[junit4] > at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:119)
[junit4] > at org.apache.lucene.store.MockDirectoryWrapper.openChecksumInput(MockDirectoryWrapper.java:1072)
[junit4] > at org.apache.lucene.codecs.lucene60.Lucene60FieldInfosFormat.read(Lucene60FieldInfosFormat.java:113)
[junit4] > at org.apache.lucene.index.SegmentReader.initFieldInfos(SegmentReader.java:190)
[junit4] > at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:93)
[junit4] > at org.apache.lucene.index.ReadersAndUpdates.writeFieldUpdates(ReadersAndUpdates.java:688)
[junit4] > at org.apache.lucene.index.IndexWriter$ReaderPool.writeSomeDocValuesUpdates(IndexWriter.java:703)
[junit4] > at org.apache.lucene.index.FrozenBufferedUpdates.apply(FrozenBufferedUpdates.java:331)
[junit4] > at org.apache.lucene.index.DocumentsWriter$ResolveUpdatesEvent.process(DocumentsWriter.java:723)
[junit4] > at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5060)
[junit4] > at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5048)
[junit4] > at org.apache.lucene.index.IndexWriter.updateDocValues(IndexWriter.java:1879)
[junit4] > at org.apache.lucene.index.TestNumericDocValuesUpdates.testTonsOfUpdates(TestNumericDocValuesUpdates.java:1557)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/build/core/test/J1/temp/lucene.index.TestNumericDocValuesUpdates_401AC1DFEE4FDDDA-001
[junit4] 2> NOTE: test params are: codec=FastCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST, chunkSize=7, maxDocsPerChunk=938, blockSize=873), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST, chunkSize=7, blockSize=873)), sim=RandomSimilarity(queryNorm=true): {}, locale=ga, timezone=America/Porto_Velho
[junit4] 2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=160754240,total=473432064
[junit4] 2> NOTE: All tests run in this JVM: [TestVirtualMethod, TestIndexFileDeleter, TestDeterminism, TestLRUQueryCache, TestQueryBuilder, TestMultiTermConstantScore, TestPriorityQueue, TestBlockPostingsFormat3, TestTragicIndexWriterDeadlock, TestSpanExplanations, TestDirectory, TestIndexWriterFromReader, TestIndexOrDocValuesQuery, TestSortedSetSelector, TestScoreCachingWrappingScorer, TestMutableValues, TestCustomTermFreq, TestByteSlices, TestNorms, TestIndexWriterOutOfFileDescriptors, TestTermQuery, TestPointValues, TestIndexSorting, TestDoubleValuesSource, TestTimeLimitingCollector, TestPhraseQuery, TestNumericDocValuesUpdates]
[junit4] Completed [133/454 (2!)] on J1 in 73.12s, 32 tests, 1 error <<< FAILURES!
[...truncated 11810 lines...]
[junit4] Suite: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest
[junit4] 2> 945122 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9799A1A109A32D92]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_9799A1A109A32D92-001/init-core-data-001
[junit4] 2> 945122 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9799A1A109A32D92]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
[junit4] 2> 945124 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9799A1A109A32D92]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 945153 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9799A1A109A32D92]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=None)
[junit4] 2> 945153 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9799A1A109A32D92]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /_i/
[junit4] 2> 947326 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9799A1A109A32D92]-worker) [ ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 949538 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9799A1A109A32D92]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 950122 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9799A1A109A32D92]-worker) [ ] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
[junit4] 2> 950238 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9799A1A109A32D92]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 952016 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9799A1A109A32D92]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 952294 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9799A1A109A32D92]-worker) [ ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/hdfs to ./temp/Jetty_lucene2.us.west_apache_org_39697_hdfs____.m38dew/webapp
[junit4] 2> 954934 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9799A1A109A32D92]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@lucene2-us-west.apache.org:39697
[junit4] 2> 959960 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9799A1A109A32D92]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 959964 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9799A1A109A32D92]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 960032 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9799A1A109A32D92]-worker) [ ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_37853_datanode____3pwd7d/webapp
[junit4] 2> 961014 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9799A1A109A32D92]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37853
[junit4] 2> 964965 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9799A1A109A32D92]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 964967 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9799A1A109A32D92]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 965161 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9799A1A109A32D92]-worker) [ ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_34895_datanode____.e3kv9i/webapp
[junit4] 2> 967097 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9799A1A109A32D92]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34895
[junit4] 2> 972958 ERROR (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_9799A1A109A32D92-001/tempDir-001/hdfsBaseDir/data/data3/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_9799A1A109A32D92-001/tempDir-001/hdfsBaseDir/data/data4/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:38982) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 973154 ERROR (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_9799A1A109A32D92-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_9799A1A109A32D92-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:38982) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 975157 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0xfe4be17f504ea: from storage DS-1beb9421-2dd7-412b-a3a8-aad544cd5cda node DatanodeRegistration(127.0.0.1:35750, datanodeUuid=5481252d-2e4a-481f-9874-46ce13a249b2, infoPort=46160, infoSecurePort=0, ipcPort=36743, storageInfo=lv=-56;cid=testClusterID;nsid=527038563;c=0), blocks: 0, hasStaleStorage: true, processing time: 14 msecs
[junit4] 2> 975206 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0xfe4be17fee642: from storage DS-5e07acde-bac7-4bb9-b800-fea05a2e10e6 node DatanodeRegistration(127.0.0.1:42459, datanodeUuid=a51ff1e7-8380-4d80-b67f-4e0c0178578c, infoPort=37891, infoSecurePort=0, ipcPort=43675, storageInfo=lv=-56;cid=testClusterID;nsid=527038563;c=0), blocks: 0, hasStaleStorage: true, processing time: 49 msecs
[junit4] 2> 975206 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0xfe4be17fee642: from storage DS-0e6e7065-d94e-491c-9c4e-c134a66a06ac node DatanodeRegistration(127.0.0.1:42459, datanodeUuid=a51ff1e7-8380-4d80-b67f-4e0c0178578c, infoPort=37891, infoSecurePort=0, ipcPort=43675, storageInfo=lv=-56;cid=testClusterID;nsid=527038563;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 975222 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0xfe4be17f504ea: from storage DS-b09f92a0-88c6-4a19-8c2b-958af7d26b58 node DatanodeRegistration(127.0.0.1:35750, datanodeUuid=5481252d-2e4a-481f-9874-46ce13a249b2, infoPort=46160, infoSecurePort=0, ipcPort=36743, storageInfo=lv=-56;cid=testClusterID;nsid=527038563;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 977495 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 977510 INFO (Thread-1209) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 977510 INFO (Thread-1209) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 977510 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.c.ZkTestServer start zk server on port:37588
[junit4] 2> 977565 ERROR (Thread-1209) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 977951 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 977952 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 978000 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 978014 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 978015 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 978016 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 978017 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 978048 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 978050 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 978213 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 978337 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 978354 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
[junit4] 2> 979569 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 979601 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6de09de5{/_i,null,AVAILABLE}
[junit4] 2> 979602 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@211f2871{HTTP/1.1,[http/1.1]}{127.0.0.1:35225}
[junit4] 2> 979602 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.e.j.s.Server Started @995450ms
[junit4] 2> 979602 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {replicaType=NRT, hostContext=/_i, hostPort=35225, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_9799A1A109A32D92-001/control-001/cores}
[junit4] 2> 979602 ERROR (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 979602 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.1.0
[junit4] 2> 979602 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 979602 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 979602 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-09-02T14:58:45.027Z
[junit4] 2> 979693 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 979693 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_9799A1A109A32D92-001/control-001/solr.xml
[junit4] 2> 979771 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@67742501, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 979882 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37588/solr
[junit4] 2> 980732 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [n:127.0.0.1:35225__i ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 980732 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [n:127.0.0.1:35225__i ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:35225__i
[junit4] 2> 980733 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [n:127.0.0.1:35225__i ] o.a.s.c.Overseer Overseer (id=98590020269965316-127.0.0.1:35225__i-n_0000000000) starting
[junit4] 2> 980788 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [n:127.0.0.1:35225__i ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35225__i
[junit4] 2> 980789 INFO (zkCallback-599-thread-1-processing-n:127.0.0.1:35225__i) [n:127.0.0.1:35225__i ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 981976 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [n:127.0.0.1:35225__i ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67742501
[junit4] 2> 982214 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [n:127.0.0.1:35225__i ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67742501
[junit4] 2> 982214 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [n:127.0.0.1:35225__i ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67742501
[junit4] 2> 982216 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [n:127.0.0.1:35225__i ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_9799A1A109A32D92-001/control-001/cores
[junit4] 2> 982423 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 982423 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37588/solr ready
[junit4] 2> 982486 INFO (qtp1720143881-4522) [n:127.0.0.1:35225__i ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:35225__i&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 982536 INFO (OverseerThreadFactory-1821-thread-1-processing-n:127.0.0.1:35225__i) [n:127.0.0.1:35225__i ] o.a.s.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 982834 INFO (qtp1720143881-4524) [n:127.0.0.1:35225__i ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 982835 INFO (qtp1720143881-4524) [n:127.0.0.1:35225__i ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 982996 INFO (zkCallback-599-thread-1-processing-n:127.0.0.1:35225__i) [n:127.0.0.1:35225__i ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 984095 INFO (qtp1720143881-4524) [n:127.0.0.1:35225__i c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 984588 INFO (qtp1720143881-4524) [n:127.0.0.1:35225__i c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
[junit4] 2> 985227 INFO (qtp1720143881-4524) [n:127.0.0.1:35225__i c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 985310 INFO (qtp1720143881-4524) [n:127.0.0.1:35225__i c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
[junit4] 2> 985310 INFO (qtp1720143881-4524) [n:127.0.0.1:35225__i c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67742501
[junit4] 2> 985492 INFO (qtp1720143881-4524) [n:127.0.0.1:35225__i c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://lucene2-us-west.apache.org:38982/solr_hdfs_home
[junit4] 2> 985493 INFO (qtp1720143881-4524) [n:127.0.0.1:35225__i c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 985493 INFO (qtp1720143881-4524) [n:127.0.0.1:35225__i c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 985493 INFO (qtp1720143881-4524) [n:127.0.0.1:35225__i c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_9799A1A109A32D92-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[hdfs://lucene2-us-west.apache.org:38982/solr_hdfs_home/control_collection/core_node2/data/]
[junit4] 2> 985525 INFO (qtp1720143881-4524) [n:127.0.0.1:35225__i c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:38982/solr_hdfs_home/control_collection/core_node2/data/snapshot_metadata
[junit4] 2> 985724 INFO (qtp1720143881-4524) [n:127.0.0.1:35225__i c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 985724 INFO (qtp1720143881-4524) [n:127.0.0.1:35225__i c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
[junit4] 2> 985724 INFO (qtp1720143881-4524) [n:127.0.0.1:35225__i c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Creating new global HDFS BlockCache
[junit4] 2> 989794 INFO (qtp1720143881-4524) [n:127.0.0.1:35225__i c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 989854 INFO (qtp1720143881-4524) [n:127.0.0.1:35225__i c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:38982/solr_hdfs_home/control_collection/core_node2/data
[junit4] 2> 990172 INFO (qtp1720143881-4524) [n:127.0.0.1:35225__i c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:38982/solr_hdfs_home/control_collection/core_node2/data/index
[junit4] 2> 990227 INFO (qtp1720143881-4524) [n:127.0.0.1:35225__i c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 990227 INFO (qtp1720143881-4524) [n:127.0.0.1:35225__i c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
[junit4] 2> 990339 INFO (qtp1720143881-4524) [n:127.0.0.1:35225__i c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 990340 INFO (qtp1720143881-4524) [n:127.0.0.1:35225__i c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=41, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=21.1669921875, floorSegmentMB=0.3056640625, forceMergeDeletesPctAllowed=0.7150883280443499, segmentsPerTier=3.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 993184 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:35750 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-1beb9421-2dd7-412b-a3a8-aad544cd5cda:NORMAL:127.0.0.1:35750|RBW], ReplicaUC[[DISK]DS-0e6e7065-d94e-491c-9c4e-c134a66a06ac:NORMAL:127.0.0.1:42459|RBW]]} size 0
[junit4] 2> 993440 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:42459 is added to blk_1073741825_1001 size 72
[junit4] 2> 993661 WARN (qtp1720143881-4524) [n:127.0.0.1:35225__i c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 994343 INFO (qtp1720143881-4524) [n:127.0.0.1:35225__i c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 994343 INFO (qtp1720143881-4524) [n:127.0.0.1:35225__i c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 994343 INFO (qtp1720143881-4524) [n:127.0.0.1:35225__i c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 994510 INFO (qtp1720143881-4524) [n:127.0.0.1:35225__i c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 994510 INFO (qtp1720143881-4524) [n:127.0.0.1:35225__i c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 994512 INFO (qtp1720143881-4524) [n:127.0.0.1:35225__i c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=37, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 995639 INFO (qtp1720143881-4524) [n:127.0.0.1:35225__i c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@114ed438[control_collection_shard1_replica_n1] main]
[junit4] 2> 995653 INFO (qtp1720143881-4524) [n:127.0.0.1:35225__i c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 995654 INFO (qtp1720143881-4524) [n:127.0.0.1:35225__i c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 995657 INFO (qtp1720143881-4524) [n:127.0.0.1:35225__i c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 995687 INFO (searcherExecutor-1824-thread-1-processing-n:127.0.0.1:35225__i x:control_collection_shard1_replica_n1 s:shard1 c:control_collection) [n:127.0.0.1:35225__i c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@114ed438[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 995687 INFO (qtp1720143881-4524) [n:127.0.0.1:35225__i c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577440343345856512
[junit4] 2> 995720 INFO (qtp1720143881-4524) [n:127.0.0.1:35225__i c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 995720 INFO (qtp1720143881-4524) [n:127.0.0.1:35225__i c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 995720 INFO (qtp1720143881-4524) [n:127.0.0.1:35225__i c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:35225/_i/control_collection_shard1_replica_n1/
[junit4] 2> 995720 INFO (qtp1720143881-4524) [n:127.0.0.1:35225__i c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 995720 INFO (qtp1720143881-4524) [n:127.0.0.1:35225__i c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:35225/_i/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 995720 INFO (qtp1720143881-4524) [n:127.0.0.1:35225__i c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 995722 INFO (qtp1720143881-4524) [n:127.0.0.1:35225__i c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:35225/_i/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 995787 INFO (zkCallback-599-thread-2-processing-n:127.0.0.1:35225__i) [n:127.0.0.1:35225__i ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 995897 INFO (zkCallback-599-thread-2-processing-n:127.0.0.1:35225__i) [n:127.0.0.1:35225__i ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 995918 INFO (qtp1720143881-4524) [n:127.0.0.1:35225__i c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 995926 INFO (qtp1720143881-4524) [n:127.0.0.1:35225__i c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=13092
[junit4] 2> 995975 INFO (qtp1720143881-4522) [n:127.0.0.1:35225__i ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 996105 INFO (zkCallback-599-thread-2-processing-n:127.0.0.1:35225__i) [n:127.0.0.1:35225__i ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 996708 INFO (OverseerCollectionConfigSetProcessor-98590020269965316-127.0.0.1:35225__i-n_0000000000) [n:127.0.0.1:35225__i ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 996982 INFO (qtp1720143881-4522) [n:127.0.0.1:35225__i ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:35225__i&wt=javabin&version=2} status=0 QTime=14495
[junit4] 2> 997262 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 997263 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37588/solr ready
[junit4] 2> 997263 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 997264 INFO (qtp1720143881-4519) [n:127.0.0.1:35225__i ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 997282 INFO (OverseerThreadFactory-1821-thread-2-processing-n:127.0.0.1:35225__i) [n:127.0.0.1:35225__i ] o.a.s.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 997334 WARN (OverseerThreadFactory-1821-thread-2-processing-n:127.0.0.1:35225__i) [n:127.0.0.1:35225__i ] o.a.s.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
[junit4] 2> 997608 INFO (qtp1720143881-4519) [n:127.0.0.1:35225__i ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 997608 INFO (qtp1720143881-4519) [n:127.0.0.1:35225__i ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=1&wt=javabin&version=2} status=0 QTime=344
[junit4] 2> 998596 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_9799A1A109A32D92-001/shard-1-001 of type NRT
[junit4] 2> 998619 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 998696 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@21f51aa7{/_i,null,AVAILABLE}
[junit4] 2> 998696 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@48e55460{HTTP/1.1,[http/1.1]}{127.0.0.1:33921}
[junit4] 2> 998696 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.e.j.s.Server Started @1014545ms
[junit4] 2> 998696 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/_i, hostPort=33921, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_9799A1A109A32D92-001/shard-1-001/cores}
[junit4] 2> 998697 ERROR (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 998697 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.1.0
[junit4] 2> 998697 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 998697 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 998697 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-09-02T14:59:04.122Z
[junit4] 2> 998773 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 998773 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_9799A1A109A32D92-001/shard-1-001/solr.xml
[junit4] 2> 998894 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@67742501, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 998990 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37588/solr
[junit4] 2> 999206 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [n:127.0.0.1:33921__i ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 999244 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [n:127.0.0.1:33921__i ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 999254 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [n:127.0.0.1:33921__i ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33921__i
[junit4] 2> 999254 INFO (zkCallback-606-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 999254 INFO (zkCallback-599-thread-2-processing-n:127.0.0.1:35225__i) [n:127.0.0.1:35225__i ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 999305 INFO (OverseerCollectionConfigSetProcessor-98590020269965316-127.0.0.1:35225__i-n_0000000000) [n:127.0.0.1:35225__i ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 999333 INFO (zkCallback-611-thread-1-processing-n:127.0.0.1:33921__i) [n:127.0.0.1:33921__i ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 999849 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [n:127.0.0.1:33921__i ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67742501
[junit4] 2> 999978 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [n:127.0.0.1:33921__i ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67742501
[junit4] 2> 999978 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [n:127.0.0.1:33921__i ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67742501
[junit4] 2> 999992 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [n:127.0.0.1:33921__i ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_9799A1A109A32D92-001/shard-1-001/cores
[junit4] 2> 1000383 INFO (qtp1720143881-4518) [n:127.0.0.1:35225__i ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:33921__i&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1000605 INFO (OverseerThreadFactory-1821-thread-3-processing-n:127.0.0.1:35225__i) [n:127.0.0.1:35225__i ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:33921__i for creating new replica
[junit4] 2> 1000681 INFO (qtp1266108625-4578) [n:127.0.0.1:33921__i ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1000681 INFO (qtp1266108625-4578) [n:127.0.0.1:33921__i ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 1001817 INFO (qtp1266108625-4578) [n:127.0.0.1:33921__i c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 1001899 INFO (qtp1266108625-4578) [n:127.0.0.1:33921__i c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.s.IndexSchema [collection1_shard2_replica_n41] Schema name=test
[junit4] 2> 1002489 INFO (qtp1266108625-4578) [n:127.0.0.1:33921__i c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 1002571 INFO (qtp1266108625-4578) [n:127.0.0.1:33921__i c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n41' using configuration from collection collection1, trusted=true
[junit4] 2> 1002571 INFO (qtp1266108625-4578) [n:127.0.0.1:33921__i c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_n41' (registry 'solr.core.collection1.shard2.replica_n41') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67742501
[junit4] 2> 1002571 INFO (qtp1266108625-4578) [n:127.0.0.1:33921__i c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://lucene2-us-west.apache.org:38982/solr_hdfs_home
[junit4] 2> 1002584 INFO (qtp1266108625-4578) [n:127.0.0.1:33921__i c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 1002584 INFO (qtp1266108625-4578) [n:127.0.0.1:33921__i c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1002584 INFO (qtp1266108625-4578) [n:127.0.0.1:33921__i c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.SolrCore [[collection1_shard2_replica_n41] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_9799A1A109A32D92-001/shard-1-001/cores/collection1_shard2_replica_n41], dataDir=[hdfs://lucene2-us-west.apache.org:38982/solr_hdfs_home/collection1/core_node42/data/]
[junit4] 2> 1002585 INFO (qtp1266108625-4578) [n:127.0.0.1:33921__i c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:38982/solr_hdfs_home/collection1/core_node42/data/snapshot_metadata
[junit4] 2> 1002638 INFO (qtp1266108625-4578) [n:127.0.0.1:33921__i c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1002638 INFO (qtp1266108625-4578) [n:127.0.0.1:33921__i c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
[junit4] 2> 1002847 INFO (qtp1266108625-4578) [n:127.0.0.1:33921__i c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1002896 INFO (qtp1266108625-4578) [n:127.0.0.1:33921__i c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:38982/solr_hdfs_home/collection1/core_node42/data
[junit4] 2> 1003072 INFO (qtp1266108625-4578) [n:127.0.0.1:33921__i c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:38982/solr_hdfs_home/collection1/core_node42/data/index
[junit4] 2> 1003077 INFO (qtp1266108625-4578) [n:127.0.0.1:33921__i c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1003077 INFO (qtp1266108625-4578) [n:127.0.0.1:33921__i c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
[junit4] 2> 1003203 INFO (qtp1266108625-4578) [n:127.0.0.1:33921__i c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1003203 INFO (qtp1266108625-4578) [n:127.0.0.1:33921__i c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=41, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=21.1669921875, floorSegmentMB=0.3056640625, forceMergeDeletesPctAllowed=0.7150883280443499, segmentsPerTier=3.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 1003775 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:42459 is added to blk_1073741826_1002{UCState=COMMITTED, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-b09f92a0-88c6-4a19-8c2b-958af7d26b58:NORMAL:127.0.0.1:35750|RBW], ReplicaUC[[DISK]DS-5e07acde-bac7-4bb9-b800-fea05a2e10e6:NORMAL:127.0.0.1:42459|RBW]]} size 72
[junit4] 2> 1003790 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:35750 is added to blk_1073741826_1002 size 72
[junit4] 2> 1004260 WARN (qtp1266108625-4578) [n:127.0.0.1:33921__i c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1004778 INFO (qtp1266108625-4578) [n:127.0.0.1:33921__i c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 1004778 INFO (qtp1266108625-4578) [n:127.0.0.1:33921__i c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1004778 INFO (qtp1266108625-4578) [n:127.0.0.1:33921__i c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 1004950 INFO (qtp1266108625-4578) [n:127.0.0.1:33921__i c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1004950 INFO (qtp1266108625-4578) [n:127.0.0.1:33921__i c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1004965 INFO (qtp1266108625-4578) [n:127.0.0.1:33921__i c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=37, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 1005127 INFO (qtp1266108625-4578) [n:127.0.0.1:33921__i c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.s.SolrIndexSearcher Opening [Searcher@30417a42[collection1_shard2_replica_n41] main]
[junit4] 2> 1005161 INFO (qtp1266108625-4578) [n:127.0.0.1:33921__i c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1005162 INFO (qtp1266108625-4578) [n:127.0.0.1:33921__i c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1005162 INFO (qtp1266108625-4578) [n:127.0.0.1:33921__i c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1005197 INFO (qtp1266108625-4578) [n:127.0.0.1:33921__i c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577440353317814272
[junit4] 2> 1005205 INFO (searcherExecutor-1835-thread-1-processing-n:127.0.0.1:33921__i x:collection1_shard2_replica_n41 s:shard2 c:collection1) [n:127.0.0.1:33921__i c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.SolrCore [collection1_shard2_replica_n41] Registered new searcher Searcher@30417a42[collection1_shard2_replica_n41] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1005214 INFO (qtp1266108625-4578) [n:127.0.0.1:33921__i c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1005214 INFO (qtp1266108625-4578) [n:127.0.0.1:33921__i c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1005214 INFO (qtp1266108625-4578) [n:127.0.0.1:33921__i c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:33921/_i/collection1_shard2_replica_n41/
[junit4] 2> 1005214 INFO (qtp1266108625-4578) [n:127.0.0.1:33921__i c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1005214 INFO (qtp1266108625-4578) [n:127.0.0.1:33921__i c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.SyncStrategy http://127.0.0.1:33921/_i/collection1_shard2_replica_n41/ has no replicas
[junit4] 2> 1005214 INFO (qtp1266108625-4578) [n:127.0.0.1:33921__i c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 1005216 INFO (qtp1266108625-4578) [n:127.0.0.1:33921__i c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:33921/_i/collection1_shard2_replica_n41/ shard2
[junit4] 2> 1005593 INFO (qtp1266108625-4578) [n:127.0.0.1:33921__i c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1005600 INFO (qtp1266108625-4578) [n:127.0.0.1:33921__i c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=4918
[junit4] 2> 1005640 INFO (qtp1720143881-4518) [n:127.0.0.1:35225__i ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:33921__i&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2} status=0 QTime=5257
[junit4] 2> 1006429 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_9799A1A109A32D92-001/shard-2-001 of type NRT
[junit4] 2> 1006429 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 1006431 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@42127ef0{/_i,null,AVAILABLE}
[junit4] 2> 1006431 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@18ec7e2d{HTTP/1.1,[http/1.1]}{127.0.0.1:33434}
[junit4] 2> 1006431 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.e.j.s.Server Started @1022280ms
[junit4] 2> 1006431 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml, hostContext=/_i, hostPort=33434, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_9799A1A109A32D92-001/shard-2-001/cores}
[junit4] 2> 1006432 ERROR (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1006446 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.1.0
[junit4] 2> 1006446 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1006446 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 1006446 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-09-02T14:59:11.871Z
[junit4] 2> 1006538 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1006538 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_9799A1A109A32D92-001/shard-2-001/solr.xml
[junit4] 2> 1006685 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@67742501, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 1006701 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37588/solr
[junit4] 2> 1006806 INFO (OverseerCollectionConfigSetProcessor-98590020269965316-127.0.0.1:35225__i-n_0000000000) [n:127.0.0.1:35225__i ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1006998 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [n:127.0.0.1:33434__i ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1006999 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [n:127.0.0.1:33434__i ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1007000 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [n:127.0.0.1:33434__i ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33434__i
[junit4] 2> 1007027 INFO (zkCallback-606-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1007027 INFO (zkCallback-599-thread-2-processing-n:127.0.0.1:35225__i) [n:127.0.0.1:35225__i ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1007046 INFO (zkCallback-611-thread-1-processing-n:127.0.0.1:33921__i) [n:127.0.0.1:33921__i ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1007046 INFO (zkCallback-617-thread-1-processing-n:127.0.0.1:33434__i) [n:127.0.0.1:33434__i ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1007479 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [n:127.0.0.1:33434__i ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67742501
[junit4] 2> 1007534 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [n:127.0.0.1:33434__i ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67742501
[junit4] 2> 1007534 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [n:127.0.0.1:33434__i ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67742501
[junit4] 2> 1007566 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[9799A1A109A32D92]) [n:127.0.0.1:33434__i ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_9799A1A109A32D92-001/shard-2-001/cores
[junit4] 2> 1007961 INFO (qtp847692449-4616) [n:127.0.0.1:33434__i ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:33434__i&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1007978 INFO (OverseerThreadFactory-1821-thread-4-processing-n:127.0.0.1:35225__i) [n:127.0.0.1:35225__i ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:33434__i for creating new replica
[junit4] 2> 1008013 INFO (qtp847692449-4614) [n:127.0.0.1:33434__i ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n43&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1008014 INFO (qtp847692449-4614) [n:127.0.0.1:33434__i ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 1009140 INFO (qtp847692449-4614) [n:127.0.0.1:33434__i c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 1009401 INFO (qtp847692449-4614) [n:127.0.0.1:33434__i c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.s.IndexSchema [collection1_shard1_replica_n43] Schema name=test
[junit4] 2> 1009954 INFO (qtp847692449-4614) [n:127.0.0.1:33434__i c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 1010084 INFO (qtp847692449-4614) [n:127.0.0.1:33434__i c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n43' using configuration from collection collection1, trusted=true
[junit4] 2> 1010084 INFO (qtp847692449-4614) [n:127.0.0.1:33434__i c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n43' (registry 'solr.core.collection1.shard1.replica_n43') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67742501
[junit4] 2> 1010084 INFO (qtp847692449-4614) [n:127.0.0.1:33434__i c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://lucene2-us-west.apache.org:38982/solr_hdfs_home
[junit4] 2> 1010084 INFO (qtp847692449-4614) [n:127.0.0.1:33434__i c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 1010084 INFO (qtp847692449-4614) [n:127.0.0.1:33434__i c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1010084 INFO (qtp847692449-4614) [n:127.0.0.1:33434__i c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.SolrCore [[collection1_shard1_replica_n43] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_9799A1A109A32D92-001/shard-2-001/cores/collection1_shard1_replica_n43], dataDir=[hdfs://lucene2-us-west.apache.org:38982/solr_hdfs_home/collection1/core_node44/data/]
[junit4] 2> 1010086 INFO (qtp847692449-4614) [n:127.0.0.1:33434__i c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:38982/solr_hdfs_home/collection1/core_node44/data/snapshot_metadata
[junit4] 2> 1010280 INFO (qtp847692449-4614) [n:127.0.0.1:33434__i c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1010280 INFO (qtp847692449-4614) [n:127.0.0.1:33434__i c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
[junit4] 2> 1010352 INFO (qtp847692449-4614) [n:127.0.0.1:33434__i c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1010368 INFO (qtp847692449-4614) [n:127.0.0.1:33434__i c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:38982/solr_hdfs_home/collection1/core_node44/data
[junit4] 2> 1010532 INFO (qtp847692449-4614) [n:127.0.0.1:33434__i c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:38982/solr_hdfs_home/collection1/core_node44/data/index
[junit4] 2> 1010600 INFO (qtp847692449-4614) [n:127.0.0.1:33434__i c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 1010600 INFO (qtp847692449-4614) [n:127.0.0.1:33434__i c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
[junit4] 2> 1010665 INFO (qtp847692449-4614) [n:127.0.0.1:33434__i c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 1010681 INFO (qtp847692449-4614) [n:127.0.0.1:33434__i c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=41, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=21.1669921875, floorSegmentMB=0.3056640625, forceMergeDeletesPctAllowed=0.7150883280443499, segmentsPerTier=3.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 1011216 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:35750 is added to blk_1073741827_1003{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-0e6e7065-d94e-491c-9c4e-c134a66a06ac:NORMAL:127.0.0.1:42459|RBW], ReplicaUC[[DISK]DS-1beb9421-2dd7-412b-a3a8-aad544cd5cda:NORMAL:127.0.0.1:35750|RBW]]} size 0
[junit4] 2> 1011246 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:42459 is added to blk_1073741827_1003{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-0e6e7065-d94e-491c-9c4e-c134a66a06ac:NORMAL:127.0.0.1:42459|RBW], ReplicaUC[[DISK]DS-1beb9421-2dd7-412b-a3a8-aad544cd5cda:NORMAL:127.0.0.1:35750|RBW]]} size 0
[junit4] 2> 1011394 WARN (qtp847692449-4614) [n:127.0.0.1:33434__i c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1012018 INFO (qtp847692449-4614) [n:127.0.0.1:33434__i c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 1012018 INFO (qtp847692449-4614) [n:127.0.0.1:33434__i c:collection1 s:shard1 x:collection1_shard1_replica
[...truncated too long message...]
5 /solr/overseer/queue-work
[junit4] 2>
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=SharedFSAutoReplicaFailoverTest -Dtests.method=test -Dtests.seed=9799A1A109A32D92 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=ar-IQ -Dtests.timezone=Australia/Tasmania -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] FAILURE 438s J2 | SharedFSAutoReplicaFailoverTest.test <<<
[junit4] > Throwable #1: java.lang.AssertionError: Timeout waiting for all live and active
[junit4] > at __randomizedtesting.SeedInfo.seed([9799A1A109A32D92:1FCD9E7BA75F406A]:0)
[junit4] > at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.testBasics(SharedFSAutoReplicaFailoverTest.java:317)
[junit4] > at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test(SharedFSAutoReplicaFailoverTest.java:136)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 1414119 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9799A1A109A32D92]-worker) [ ] o.a.h.h.s.d.DirectoryScanner DirectoryScanner: shutdown has been called
[junit4] 2> 1414393 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9799A1A109A32D92]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
[junit4] 2> 1414512 WARN (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_9799A1A109A32D92-001/tempDir-001/hdfsBaseDir/data/data3/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_9799A1A109A32D92-001/tempDir-001/hdfsBaseDir/data/data4/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:38982) [ ] o.a.h.h.s.d.IncrementalBlockReportManager IncrementalBlockReportManager interrupted
[junit4] 2> 1414512 WARN (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_9799A1A109A32D92-001/tempDir-001/hdfsBaseDir/data/data3/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_9799A1A109A32D92-001/tempDir-001/hdfsBaseDir/data/data4/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:38982) [ ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-640428223-127.0.0.1-1504364293883 (Datanode Uuid a51ff1e7-8380-4d80-b67f-4e0c0178578c) service to lucene2-us-west.apache.org/127.0.0.1:38982
[junit4] 2> 1414524 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9799A1A109A32D92]-worker) [ ] o.a.h.h.s.d.DirectoryScanner DirectoryScanner: shutdown has been called
[junit4] 2> 1414687 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9799A1A109A32D92]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
[junit4] 2> 1414707 WARN (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_9799A1A109A32D92-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_9799A1A109A32D92-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:38982) [ ] o.a.h.h.s.d.IncrementalBlockReportManager IncrementalBlockReportManager interrupted
[junit4] 2> 1414721 WARN (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_9799A1A109A32D92-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_9799A1A109A32D92-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:38982) [ ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-640428223-127.0.0.1-1504364293883 (Datanode Uuid 5481252d-2e4a-481f-9874-46ce13a249b2) service to lucene2-us-west.apache.org/127.0.0.1:38982
[junit4] 2> 1414987 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[9799A1A109A32D92]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@lucene2-us-west.apache.org:0
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_9799A1A109A32D92-001
[junit4] 2> Sep 02, 2017 3:06:01 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 34 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {id=Lucene50(blocksize=128), text=PostingsFormat(name=Memory)}, docValues:{_version_=DocValuesFormat(name=Lucene70)}, maxPointsInLeafNode=1538, maxMBSortInHeap=7.059506632002865, sim=RandomSimilarity(queryNorm=false): {}, locale=ar-IQ, timezone=Australia/Tasmania
[junit4] 2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=5,free=52953728,total=515375104
[junit4] 2> NOTE: All tests run in this JVM: [VMParamsZkACLAndCredentialsProvidersTest, TestLegacyNumericRangeQueryBuilder, MigrateRouteKeyTest, XsltUpdateRequestHandlerTest, ZkStateWriterTest, DistributedFacetPivotSmallTest, HighlighterMaxOffsetTest, TestStressReorder, DocExpirationUpdateProcessorFactoryTest, SolrCoreCheckLockOnStartupTest, SparseHLLTest, DeleteLastCustomShardedReplicaTest, TestGraphTermsQParserPlugin, DateFieldTest, TestRecovery, RecoveryZkTest, OverseerStatusTest, TestSSLRandomization, PolyFieldTest, TestCorePropertiesReload, TestNestedDocsSort, TestGroupingSearch, TestFieldCollectionResource, TestUniqueKeyFieldResource, UniqFieldsUpdateProcessorFactoryTest, TestCodecSupport, TestUtils, PKIAuthenticationIntegrationTest, OverseerCollectionConfigSetProcessorTest, TestAnalyzedSuggestions, AssignTest, CreateCollectionCleanupTest, TestLRUStatsCache, TestRuleBasedAuthorizationPlugin, SpatialHeatmapFacetsTest, ResponseLogComponentTest, TestIndexSearcher, PreAnalyzedFieldManagedSchemaCloudTest, TestNamedUpdateProcessors, TestReversedWildcardFilterFactory, SolrIndexMetricsTest, InfixSuggestersTest, TestLMJelinekMercerSimilarityFactory, SharedFSAutoReplicaFailoverTest]
[junit4] Completed [121/731 (1!)] on J2 in 479.13s, 1 test, 1 failure <<< FAILURES!
[...truncated 37056 lines...]
-ecj-javadoc-lint-src:
[mkdir] Created dir: /tmp/ecj818560437
[ecj-lint] Compiling 30 source files to /tmp/ecj818560437
[ecj-lint] ----------
[ecj-lint] 1. ERROR in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/replicator/src/java/org/apache/lucene/replicator/nrt/PrimaryNode.java (at line 33)
[ecj-lint] import org.apache.lucene.index.Term;
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] The import org.apache.lucene.index.Term is never used
[ecj-lint] ----------
[ecj-lint] 2. ERROR in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/replicator/src/java/org/apache/lucene/replicator/nrt/PrimaryNode.java (at line 37)
[ecj-lint] import org.apache.lucene.search.TermQuery;
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] The import org.apache.lucene.search.TermQuery is never used
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 3. ERROR in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/replicator/src/java/org/apache/lucene/replicator/nrt/ReplicaNode.java (at line 36)
[ecj-lint] import org.apache.lucene.index.DirectoryReader;
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] The import org.apache.lucene.index.DirectoryReader is never used
[ecj-lint] ----------
[ecj-lint] 3 problems (3 errors)
BUILD FAILED
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/build.xml:818: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/build.xml:101: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/build.xml:208: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/common-build.xml:2205: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/common-build.xml:2030: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/common-build.xml:2069: Compile failed; see the compiler error output for details.
Total time: 718 minutes 21 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 35 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/35/
7 tests failed.
FAILED: org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([98D7CDCF196793D5]:0)
FAILED: junit.framework.TestSuite.org.apache.lucene.spatial3d.TestGeo3DPoint
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([98D7CDCF196793D5]:0)
FAILED: org.apache.solr.cloud.CdcrReplicationHandlerTest.testReplicationWithBufferedUpdates
Error Message:
Timeout while trying to assert number of documents @ source_collection
Stack Trace:
java.lang.AssertionError: Timeout while trying to assert number of documents @ source_collection
at __randomizedtesting.SeedInfo.seed([414FC989A1A4EA57:92469997E43776C0]:0)
at org.apache.solr.cloud.BaseCdcrDistributedZkTest.assertNumDocs(BaseCdcrDistributedZkTest.java:282)
at org.apache.solr.cloud.CdcrReplicationHandlerTest.testReplicationWithBufferedUpdates(CdcrReplicationHandlerTest.java:233)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.AssertionError: expected:<200> but was:<210>
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.BaseCdcrDistributedZkTest.assertNumDocs(BaseCdcrDistributedZkTest.java:273)
... 42 more
FAILED: org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest.test
Error Message:
The Monkey ran for over 45 seconds and no jetties were stopped - this is worth investigating!
Stack Trace:
java.lang.AssertionError: The Monkey ran for over 45 seconds and no jetties were stopped - this is worth investigating!
at __randomizedtesting.SeedInfo.seed([414FC989A1A4EA57:C91BF6530F5887AF]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.ChaosMonkey.stopTheMonkey(ChaosMonkey.java:587)
at org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest.test(ChaosMonkeySafeLeaderWithPullReplicasTest.java:174)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest
Error Message:
6 threads leaked from SUITE scope at org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest: 1) Thread[id=110522, name=StoppableCommitThread-EventThread, state=WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501) 2) Thread[id=110574, name=zkCallback-29781-thread-2, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=110523, name=zkCallback-29781-thread-1, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=110472, name=StoppableCommitThread, state=WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:502) at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1406) at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1210) at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:357) at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:354) at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60) at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:354) at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1127) at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1113) at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:648) at org.apache.solr.client.solrj.impl.CloudSolrClient.getDocCollection(CloudSolrClient.java:1227) at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:827) at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:793) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178) at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:524) at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:544) at org.apache.solr.cloud.StoppableCommitThread.run(StoppableCommitThread.java:49) 5) Thread[id=110521, name=StoppableCommitThread-SendThread(127.0.0.1:33460), state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060) 6) Thread[id=110575, name=zkCallback-29781-thread-3, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 6 threads leaked from SUITE scope at org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest:
1) Thread[id=110522, name=StoppableCommitThread-EventThread, state=WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)
2) Thread[id=110574, name=zkCallback-29781-thread-2, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=110523, name=zkCallback-29781-thread-1, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
4) Thread[id=110472, name=StoppableCommitThread, state=WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:502)
at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1406)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1210)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:357)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:354)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:354)
at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1127)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1113)
at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:648)
at org.apache.solr.client.solrj.impl.CloudSolrClient.getDocCollection(CloudSolrClient.java:1227)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:827)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:793)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:524)
at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:544)
at org.apache.solr.cloud.StoppableCommitThread.run(StoppableCommitThread.java:49)
5) Thread[id=110521, name=StoppableCommitThread-SendThread(127.0.0.1:33460), state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)
6) Thread[id=110575, name=zkCallback-29781-thread-3, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([414FC989A1A4EA57]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=110574, name=zkCallback-29781-thread-2, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=110523, name=zkCallback-29781-thread-1, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=110521, name=StoppableCommitThread-SendThread(127.0.0.1:33460), state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060) 4) Thread[id=110575, name=zkCallback-29781-thread-3, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=110574, name=zkCallback-29781-thread-2, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=110523, name=zkCallback-29781-thread-1, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=110521, name=StoppableCommitThread-SendThread(127.0.0.1:33460), state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)
4) Thread[id=110575, name=zkCallback-29781-thread-3, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([414FC989A1A4EA57]:0)
FAILED: org.apache.solr.cloud.CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster
Error Message:
Document mismatch on target after sync expected:<20000> but was:<10100>
Stack Trace:
java.lang.AssertionError: Document mismatch on target after sync expected:<20000> but was:<10100>
at __randomizedtesting.SeedInfo.seed([414FC989A1A4EA57:950A82D046F259AC]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.apache.solr.cloud.CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster(CdcrBootstrapTest.java:309)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 10514 lines...]
[junit4] Suite: org.apache.lucene.spatial3d.TestGeo3DPoint
[junit4] 2> avg 29, 2017 12:03:46 AM com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
[junit4] 2> WARNING: Suite execution timed out: org.apache.lucene.spatial3d.TestGeo3DPoint
[junit4] 2> 1) Thread[id=1, name=main, state=WAITING, group=main]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Thread.join(Thread.java:1252)
[junit4] 2> at java.lang.Thread.join(Thread.java:1326)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:608)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:457)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:251)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:368)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:13)
[junit4] 2> 2) Thread[id=11, name=SUITE-TestGeo3DPoint-seed#[98D7CDCF196793D5], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at java.lang.Thread.getStackTrace(Thread.java:1559)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:696)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:693)
[junit4] 2> at java.security.AccessController.doPrivileged(Native Method)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getStackTrace(ThreadLeakControl.java:693)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getThreadsWithTraces(ThreadLeakControl.java:709)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:689)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:65)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:415)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:678)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:140)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:598)
[junit4] 2> 3) Thread[id=12, name=TEST-TestGeo3DPoint.testRandomBig-seed#[98D7CDCF196793D5], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at org.apache.lucene.spatial3d.geom.Plane.findIntersectionBounds(Plane.java:1053)
[junit4] 2> at org.apache.lucene.spatial3d.geom.Plane.recordBounds(Plane.java:1214)
[junit4] 2> at org.apache.lucene.spatial3d.geom.XYZBounds.addIntersection(XYZBounds.java:262)
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoStandardPath$PathSegment.getBounds(GeoStandardPath.java:872)
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoStandardPath.getBounds(GeoStandardPath.java:336)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.verify(TestGeo3DPoint.java:846)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.doTestRandom(TestGeo3DPoint.java:548)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig(TestGeo3DPoint.java:480)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 4) Thread[id=9, name=JUnit4-serializer-daemon, state=TIMED_WAITING, group=main]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:50)
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestGeo3DPoint -Dtests.method=testRandomBig -Dtests.seed=98D7CDCF196793D5 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=sr-Latn-BA -Dtests.timezone=US/Eastern -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 6654s J1 | TestGeo3DPoint.testRandomBig <<<
[junit4] > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
[junit4] > at __randomizedtesting.SeedInfo.seed([98D7CDCF196793D5]:0)
[junit4] 2> avg 29, 2017 12:03:46 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> avg 29, 2017 12:04:06 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.lucene.spatial3d.TestGeo3DPoint:
[junit4] 2> 1) Thread[id=12, name=TEST-TestGeo3DPoint.testRandomBig-seed#[98D7CDCF196793D5], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at org.apache.lucene.spatial3d.geom.XYZBounds.addPlane(XYZBounds.java:161)
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoStandardPath$SegmentEndpoint.getBounds(GeoStandardPath.java:609)
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoStandardPath.getBounds(GeoStandardPath.java:339)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.verify(TestGeo3DPoint.java:846)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.doTestRandom(TestGeo3DPoint.java:548)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig(TestGeo3DPoint.java:480)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> avg 29, 2017 12:04:06 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=12, name=TEST-TestGeo3DPoint.testRandomBig-seed#[98D7CDCF196793D5], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/build/spatial3d/test/J1/temp/lucene.spatial3d.TestGeo3DPoint_98D7CDCF196793D5-001
[junit4] 2> Aug 29, 2017 4:04:06 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: All leaked threads terminated.
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {id=PostingsFormat(name=LuceneFixedGap)}, docValues:{id=DocValuesFormat(name=Lucene70), point=DocValuesFormat(name=Asserting)}, maxPointsInLeafNode=1018, maxMBSortInHeap=6.416690269127653, sim=RandomSimilarity(queryNorm=true): {}, locale=sr-Latn-BA, timezone=US/Eastern
[junit4] 2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=39527016,total=389021696
[junit4] 2> NOTE: All tests run in this JVM: [TestGeo3DPoint]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestGeo3DPoint -Dtests.seed=98D7CDCF196793D5 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=sr-Latn-BA -Dtests.timezone=US/Eastern -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J1 | TestGeo3DPoint (suite) <<<
[junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4] > at __randomizedtesting.SeedInfo.seed([98D7CDCF196793D5]:0)
[junit4] Completed [14/15 (1!)] on J1 in 7221.96s, 11 tests, 2 errors <<< FAILURES!
[...truncated 1947 lines...]
[junit4] Suite: org.apache.solr.cloud.CdcrBootstrapTest
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.CdcrBootstrapTest_414FC989A1A4EA57-001/init-core-data-001
[junit4] 2> 188486 WARN (SUITE-CdcrBootstrapTest-seed#[414FC989A1A4EA57]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
[junit4] 2> 188586 ERROR (Thread-74) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 188917 ERROR (jetty-launcher-126-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 1> Target zkHost = 127.0.0.1:42371/solr
[junit4] 2> 191207 ERROR (Thread-84) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 191454 ERROR (jetty-launcher-138-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 1> Adding 100 docs with commit=true, numDocs=100
[junit4] 1> Adding 100 docs with commit=true, numDocs=200
[junit4] 1> Adding 100 docs with commit=true, numDocs=300
[junit4] 1> Adding 100 docs with commit=true, numDocs=400
[junit4] 1> Adding 100 docs with commit=true, numDocs=500
[junit4] 1> Adding 100 docs with commit=true, numDocs=600
[junit4] 1> Adding 100 docs with commit=true, numDocs=700
[junit4] 1> Adding 100 docs with commit=true, numDocs=800
[junit4] 1> Adding 100 docs with commit=true, numDocs=900
[junit4] 1> Adding 100 docs with commit=true, numDocs=1000
[junit4] 1> Adding 100 docs with commit=true, numDocs=1100
[junit4] 1> Adding 100 docs with commit=true, numDocs=1200
[junit4] 1> Adding 100 docs with commit=true, numDocs=1300
[junit4] 1> Adding 100 docs with commit=true, numDocs=1400
[junit4] 1> Adding 100 docs with commit=true, numDocs=1500
[junit4] 1> Adding 100 docs with commit=true, numDocs=1600
[junit4] 1> Adding 100 docs with commit=true, numDocs=1700
[junit4] 1> Adding 100 docs with commit=true, numDocs=1800
[junit4] 1> Adding 100 docs with commit=true, numDocs=1900
[junit4] 1> Adding 100 docs with commit=true, numDocs=2000
[junit4] 1> Adding 100 docs with commit=true, numDocs=2100
[junit4] 1> Adding 100 docs with commit=true, numDocs=2200
[junit4] 1> Adding 100 docs with commit=true, numDocs=2300
[junit4] 1> Adding 100 docs with commit=true, numDocs=2400
[junit4] 1> Adding 100 docs with commit=true, numDocs=2500
[junit4] 1> Adding 100 docs with commit=true, numDocs=2600
[junit4] 1> Adding 100 docs with commit=true, numDocs=2700
[junit4] 1> Adding 100 docs with commit=true, numDocs=2800
[junit4] 1> Adding 100 docs with commit=true, numDocs=2900
[junit4] 1> Adding 100 docs with commit=true, numDocs=3000
[junit4] 1> Adding 100 docs with commit=true, numDocs=3100
[junit4] 1> Adding 100 docs with commit=true, numDocs=3200
[junit4] 1> Adding 100 docs with commit=true, numDocs=3300
[junit4] 1> Adding 100 docs with commit=true, numDocs=3400
[junit4] 1> Adding 100 docs with commit=true, numDocs=3500
[junit4] 1> Adding 100 docs with commit=true, numDocs=3600
[junit4] 1> Adding 100 docs with commit=true, numDocs=3700
[junit4] 1> Adding 100 docs with commit=true, numDocs=3800
[junit4] 1> Adding 100 docs with commit=true, numDocs=3900
[junit4] 1> Adding 100 docs with commit=true, numDocs=4000
[junit4] 1> Adding 100 docs with commit=true, numDocs=4100
[junit4] 1> Adding 100 docs with commit=true, numDocs=4200
[junit4] 1> Adding 100 docs with commit=true, numDocs=4300
[junit4] 1> Adding 100 docs with commit=true, numDocs=4400
[junit4] 1> Adding 100 docs with commit=true, numDocs=4500
[junit4] 1> Adding 100 docs with commit=true, numDocs=4600
[junit4] 1> Adding 100 docs with commit=true, numDocs=4700
[junit4] 1> Adding 100 docs with commit=true, numDocs=4800
[junit4] 1> Adding 100 docs with commit=true, numDocs=4900
[junit4] 1> Adding 100 docs with commit=true, numDocs=5000
[junit4] 1> Adding 100 docs with commit=true, numDocs=5100
[junit4] 1> Adding 100 docs with commit=true, numDocs=5200
[junit4] 1> Adding 100 docs with commit=true, numDocs=5300
[junit4] 1> Adding 100 docs with commit=true, numDocs=5400
[junit4] 1> Adding 100 docs with commit=true, numDocs=5500
[junit4] 1> Adding 100 docs with commit=true, numDocs=5600
[junit4] 1> Adding 100 docs with commit=true, numDocs=5700
[junit4] 1> Adding 100 docs with commit=true, numDocs=5800
[junit4] 1> Adding 100 docs with commit=true, numDocs=5900
[junit4] 1> Adding 100 docs with commit=true, numDocs=6000
[junit4] 1> Adding 100 docs with commit=true, numDocs=6100
[junit4] 1> Adding 100 docs with commit=true, numDocs=6200
[junit4] 1> Adding 100 docs with commit=true, numDocs=6300
[junit4] 1> Adding 100 docs with commit=true, numDocs=6400
[junit4] 1> Adding 100 docs with commit=true, numDocs=6500
[junit4] 1> Adding 100 docs with commit=true, numDocs=6600
[junit4] 1> Adding 100 docs with commit=true, numDocs=6700
[junit4] 1> Adding 100 docs with commit=true, numDocs=6800
[junit4] 1> Adding 100 docs with commit=true, numDocs=6900
[junit4] 1> Adding 100 docs with commit=true, numDocs=7000
[junit4] 1> Adding 100 docs with commit=true, numDocs=7100
[junit4] 1> Adding 100 docs with commit=true, numDocs=7200
[junit4] 1> Adding 100 docs with commit=true, numDocs=7300
[junit4] 1> Adding 100 docs with commit=true, numDocs=7400
[junit4] 1> Adding 100 docs with commit=true, numDocs=7500
[junit4] 1> Adding 100 docs with commit=true, numDocs=7600
[junit4] 1> Adding 100 docs with commit=true, numDocs=7700
[junit4] 1> Adding 100 docs with commit=true, numDocs=7800
[junit4] 1> Adding 100 docs with commit=true, numDocs=7900
[junit4] 1> Adding 100 docs with commit=true, numDocs=8000
[junit4] 1> Adding 100 docs with commit=true, numDocs=8100
[junit4] 1> Adding 100 docs with commit=true, numDocs=8200
[junit4] 1> Adding 100 docs with commit=true, numDocs=8300
[junit4] 1> Adding 100 docs with commit=true, numDocs=8400
[junit4] 1> Adding 100 docs with commit=true, numDocs=8500
[junit4] 1> Adding 100 docs with commit=true, numDocs=8600
[junit4] 1> Adding 100 docs with commit=true, numDocs=8700
[junit4] 1> Adding 100 docs with commit=true, numDocs=8800
[junit4] 1> Adding 100 docs with commit=true, numDocs=8900
[junit4] 1> Adding 100 docs with commit=true, numDocs=9000
[junit4] 1> Adding 100 docs with commit=true, numDocs=9100
[junit4] 1> Adding 100 docs with commit=true, numDocs=9200
[junit4] 1> Adding 100 docs with commit=true, numDocs=9300
[junit4] 1> Adding 100 docs with commit=true, numDocs=9400
[junit4] 1> Adding 100 docs with commit=true, numDocs=9500
[junit4] 1> Adding 100 docs with commit=true, numDocs=9600
[junit4] 1> Adding 100 docs with commit=true, numDocs=9700
[junit4] 1> Adding 100 docs with commit=true, numDocs=9800
[junit4] 1> Adding 100 docs with commit=true, numDocs=9900
[junit4] 1> Adding 100 docs with commit=true, numDocs=10000
[junit4] 1> Cdcr queue response: {responseHeader={status=0,QTime=31},queues={127.0.0.1:42371/solr={cdcr-target={queueSize=10100,lastTimestamp=}}},tlogTotalSize=494631,tlogTotalCount=100,updateLogSynchronizer=stopped}
[junit4] 2> 271427 ERROR (updateExecutor-130-thread-1-processing-n:127.0.0.1:37463_solr x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node2) [n:127.0.0.1:37463_solr c:cdcr-target s:shard1 r:core_node2 x:cdcr-target_shard1_replica_n1] o.a.s.h.CdcrRequestHandler Bootstrap operation failed
[junit4] 2> java.util.concurrent.ExecutionException: java.lang.AssertionError
[junit4] 2> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
[junit4] 2> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
[junit4] 2> at org.apache.solr.handler.CdcrRequestHandler.lambda$handleBootstrapAction$0(CdcrRequestHandler.java:646)
[junit4] 2> at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: java.lang.AssertionError
[junit4] 2> at org.apache.solr.handler.CdcrRequestHandler$BootstrapCallable.call(CdcrRequestHandler.java:789)
[junit4] 2> at org.apache.solr.handler.CdcrRequestHandler$BootstrapCallable.call(CdcrRequestHandler.java:716)
[junit4] 2> at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
[junit4] 2> ... 5 more
[junit4] 2> 272557 WARN (cdcr-bootstrap-status-157-thread-1-processing-n:127.0.0.1:40002_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node2) [n:127.0.0.1:40002_solr c:cdcr-source s:shard1 r:core_node2 x:cdcr-source_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager Cancelling waiting for bootstrap on target: cdcr-target shard: shard1 to complete
[junit4] 1> Adding 100 docs with commit=true, numDocs=10100
[junit4] 1> Adding 100 docs with commit=true, numDocs=10200
[junit4] 1> Adding 100 docs with commit=true, numDocs=10300
[junit4] 1> Adding 100 docs with commit=true, numDocs=10400
[junit4] 1> Adding 100 docs with commit=true, numDocs=10500
[junit4] 1> Adding 100 docs with commit=true, numDocs=10600
[junit4] 1> Adding 100 docs with commit=true, numDocs=10700
[junit4] 1> Adding 100 docs with commit=true, numDocs=10800
[junit4] 1> Adding 100 docs with commit=true, numDocs=10900
[junit4] 1> Adding 100 docs with commit=true, numDocs=11000
[junit4] 2> 287133 WARN (cdcr-bootstrap-status-160-thread-1-processing-n:127.0.0.1:40002_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node2) [n:127.0.0.1:40002_solr c:cdcr-source s:shard1 r:core_node2 x:cdcr-source_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager Cancelling waiting for bootstrap on target: cdcr-target shard: shard1 to complete
[junit4] 2> 287377 WARN (zkCallback-145-thread-6-processing-n:127.0.0.1:40002_solr) [n:127.0.0.1:40002_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 287394 ERROR (TEST-CdcrBootstrapTest.testBootstrapWithSourceCluster-seed#[414FC989A1A4EA57]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 287671 WARN (Thread-84) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/aliases.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 7 /solr/collections/cdcr-source/state.json
[junit4] 2> 4 /solr/collections/cdcr-source/cdcr/state/process
[junit4] 2> 3 /solr/collections/cdcr-source/cdcr/state/buffer
[junit4] 2> 2 /solr/clusterstate.json
[junit4] 2> 2 /solr/clusterprops.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/live_nodes
[junit4] 2> 2 /solr/collections
[junit4] 2>
[junit4] 2> 288078 WARN (zkCallback-133-thread-3-processing-n:127.0.0.1:37463_solr) [n:127.0.0.1:37463_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 288144 ERROR (TEST-CdcrBootstrapTest.testBootstrapWithSourceCluster-seed#[414FC989A1A4EA57]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 288277 WARN (Thread-74) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/configs/cdcr-target/managed-schema
[junit4] 2> 3 /solr/aliases.json
[junit4] 2> 3 /solr/collections/cdcr-target/leaders/shard1/leader
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 6 /solr/collections/cdcr-target/state.json
[junit4] 2> 4 /solr/collections/cdcr-target/cdcr/state/process
[junit4] 2> 3 /solr/collections/cdcr-target/cdcr/state/buffer
[junit4] 2> 3 /solr/clusterstate.json
[junit4] 2> 3 /solr/clusterprops.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/live_nodes
[junit4] 2> 3 /solr/collections
[junit4] 2>
[junit4] 2> 288392 ERROR (Thread-110) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 288510 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e2c487f9d0000, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 288575 ERROR (jetty-launcher-161-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 1> Target zkHost = 127.0.0.1:41453/solr
[junit4] 2> 290546 ERROR (Thread-120) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 290725 ERROR (jetty-launcher-173-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 1> Adding 100 docs with commit=true, numDocs=100
[junit4] 1> Adding 100 docs with commit=true, numDocs=200
[junit4] 1> Adding 100 docs with commit=true, numDocs=300
[junit4] 1> Adding 100 docs with commit=true, numDocs=400
[junit4] 1> Adding 100 docs with commit=true, numDocs=500
[junit4] 1> Adding 100 docs with commit=true, numDocs=600
[junit4] 1> Adding 100 docs with commit=true, numDocs=700
[junit4] 1> Adding 100 docs with commit=true, numDocs=800
[junit4] 1> Adding 100 docs with commit=true, numDocs=900
[junit4] 1> Adding 100 docs with commit=true, numDocs=1000
[junit4] 1> Adding 100 docs with commit=true, numDocs=1100
[junit4] 1> Adding 100 docs with commit=true, numDocs=1200
[junit4] 1> Adding 100 docs with commit=true, numDocs=1300
[junit4] 1> Adding 100 docs with commit=true, numDocs=1400
[junit4] 1> Adding 100 docs with commit=true, numDocs=1500
[junit4] 1> Adding 100 docs with commit=true, numDocs=1600
[junit4] 1> Adding 100 docs with commit=true, numDocs=1700
[junit4] 1> Adding 100 docs with commit=true, numDocs=1800
[junit4] 1> Adding 100 docs with commit=true, numDocs=1900
[junit4] 1> Adding 100 docs with commit=true, numDocs=2000
[junit4] 1> Adding 100 docs with commit=true, numDocs=2100
[junit4] 1> Adding 100 docs with commit=true, numDocs=2200
[junit4] 1> Adding 100 docs with commit=true, numDocs=2300
[junit4] 1> Adding 100 docs with commit=true, numDocs=2400
[junit4] 1> Adding 100 docs with commit=true, numDocs=2500
[junit4] 1> Adding 100 docs with commit=true, numDocs=2600
[junit4] 1> Adding 100 docs with commit=true, numDocs=2700
[junit4] 1> Adding 100 docs with commit=true, numDocs=2800
[junit4] 1> Adding 100 docs with commit=true, numDocs=2900
[junit4] 1> Adding 100 docs with commit=true, numDocs=3000
[junit4] 1> Adding 100 docs with commit=true, numDocs=3100
[junit4] 1> Adding 100 docs with commit=true, numDocs=3200
[junit4] 1> Adding 100 docs with commit=true, numDocs=3300
[junit4] 1> Adding 100 docs with commit=true, numDocs=3400
[junit4] 1> Adding 100 docs with commit=true, numDocs=3500
[junit4] 1> Adding 100 docs with commit=true, numDocs=3600
[junit4] 1> Adding 100 docs with commit=true, numDocs=3700
[junit4] 1> Adding 100 docs with commit=true, numDocs=3800
[junit4] 1> Adding 100 docs with commit=true, numDocs=3900
[junit4] 1> Adding 100 docs with commit=true, numDocs=4000
[junit4] 1> Adding 100 docs with commit=true, numDocs=4100
[junit4] 1> Adding 100 docs with commit=true, numDocs=4200
[junit4] 1> Adding 100 docs with commit=true, numDocs=4300
[junit4] 1> Adding 100 docs with commit=true, numDocs=4400
[junit4] 1> Adding 100 docs with commit=true, numDocs=4500
[junit4] 1> Adding 100 docs with commit=true, numDocs=4600
[junit4] 1> Adding 100 docs with commit=true, numDocs=4700
[junit4] 1> Adding 100 docs with commit=true, numDocs=4800
[junit4] 1> Adding 100 docs with commit=true, numDocs=4900
[junit4] 1> Adding 100 docs with commit=true, numDocs=5000
[junit4] 1> Adding 100 docs with commit=true, numDocs=5100
[junit4] 1> Adding 100 docs with commit=true, numDocs=5200
[junit4] 1> Adding 100 docs with commit=true, numDocs=5300
[junit4] 1> Adding 100 docs with commit=true, numDocs=5400
[junit4] 1> Adding 100 docs with commit=true, numDocs=5500
[junit4] 1> Adding 100 docs with commit=true, numDocs=5600
[junit4] 1> Adding 100 docs with commit=true, numDocs=5700
[junit4] 1> Adding 100 docs with commit=true, numDocs=5800
[junit4] 1> Adding 100 docs with commit=true, numDocs=5900
[junit4] 1> Adding 100 docs with commit=true, numDocs=6000
[junit4] 1> Adding 100 docs with commit=true, numDocs=6100
[junit4] 1> Adding 100 docs with commit=true, numDocs=6200
[junit4] 1> Adding 100 docs with commit=true, numDocs=6300
[junit4] 1> Adding 100 docs with commit=true, numDocs=6400
[junit4] 1> Adding 100 docs with commit=true, numDocs=6500
[junit4] 1> Adding 100 docs with commit=true, numDocs=6600
[junit4] 1> Adding 100 docs with commit=true, numDocs=6700
[junit4] 1> Adding 100 docs with commit=true, numDocs=6800
[junit4] 1> Adding 100 docs with commit=true, numDocs=6900
[junit4] 1> Adding 100 docs with commit=true, numDocs=7000
[junit4] 1> Adding 100 docs with commit=true, numDocs=7100
[junit4] 1> Adding 100 docs with commit=true, numDocs=7200
[junit4] 1> Adding 100 docs with commit=true, numDocs=7300
[junit4] 1> Adding 100 docs with commit=true, numDocs=7400
[junit4] 1> Adding 100 docs with commit=true, numDocs=7500
[junit4] 1> Adding 100 docs with commit=true, numDocs=7600
[junit4] 1> Adding 100 docs with commit=true, numDocs=7700
[junit4] 1> Adding 100 docs with commit=true, numDocs=7800
[junit4] 1> Adding 100 docs with commit=true, numDocs=7900
[junit4] 1> Adding 100 docs with commit=true, numDocs=8000
[junit4] 1> Adding 100 docs with commit=true, numDocs=8100
[junit4] 1> Adding 100 docs with commit=true, numDocs=8200
[junit4] 1> Adding 100 docs with commit=true, numDocs=8300
[junit4] 1> Adding 100 docs with commit=true, numDocs=8400
[junit4] 1> Adding 100 docs with commit=true, numDocs=8500
[junit4] 1> Adding 100 docs with commit=true, numDocs=8600
[junit4] 1> Adding 100 docs with commit=true, numDocs=8700
[junit4] 1> Adding 100 docs with commit=true, numDocs=8800
[junit4] 1> Adding 100 docs with commit=true, numDocs=8900
[junit4] 1> Adding 100 docs with commit=true, numDocs=9000
[junit4] 1> Adding 100 docs with commit=true, numDocs=9100
[junit4] 1> Adding 100 docs with commit=true, numDocs=9200
[junit4] 1> Adding 100 docs with commit=true, numDocs=9300
[junit4] 1> Adding 100 docs with commit=true, numDocs=9400
[junit4] 1> Adding 100 docs with commit=true, numDocs=9500
[junit4] 1> Adding 100 docs with commit=true, numDocs=9600
[junit4] 1> Adding 100 docs with commit=true, numDocs=9700
[junit4] 1> Adding 100 docs with commit=true, numDocs=9800
[junit4] 1> Adding 100 docs with commit=true, numDocs=9900
[junit4] 1> Adding 100 docs with commit=true, numDocs=10000
[junit4] 2> 336021 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e2c4888370008, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 337288 WARN (zkCallback-180-thread-1-processing-n:127.0.0.1:45076_solr) [n:127.0.0.1:45076_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 337338 ERROR (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[414FC989A1A4EA57]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 1> -
[junit4] 1> replica:core_node2 rstate:down live:true
[junit4] 1> -
[junit4] 1> replica:core_node2 rstate:active live:true
[junit4] 1> no one is recoverying
[junit4] 2> 340547 ERROR (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[414FC989A1A4EA57]) [ ] o.a.s.c.s.i.CloudSolrClient Request to collection cdcr-source failed due to (510) org.apache.solr.common.SolrException: Could not find a healthy node to handle the request., retry? 0
[junit4] 2> 340547 WARN (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[414FC989A1A4EA57]) [ ] o.a.s.c.s.i.CloudSolrClient Re-trying request to collection(s) cdcr-source after stale state error from server.
[junit4] 1> Cdcr queue response: {responseHeader={status=0,QTime=183},queues={127.0.0.1:41453/solr={cdcr-target={queueSize=-1927282669,lastTimestamp=}}},tlogTotalSize=4973,tlogTotalCount=1,updateLogSynchronizer=stopped}
[junit4] 2> 347300 WARN (cdcr-bootstrap-status-199-thread-1-processing-n:127.0.0.1:36989_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node2) [n:127.0.0.1:36989_solr c:cdcr-source s:shard1 r:core_node2 x:cdcr-source_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager Cancelling waiting for bootstrap on target: cdcr-target shard: shard1 to complete
[junit4] 2> 347642 ERROR (updateExecutor-165-thread-1-processing-n:127.0.0.1:34947_solr x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node2) [n:127.0.0.1:34947_solr c:cdcr-target s:shard1 r:core_node2 x:cdcr-target_shard1_replica_n1] o.a.s.h.CdcrRequestHandler Bootstrap operation failed
[junit4] 2> java.util.concurrent.ExecutionException: java.lang.AssertionError
[junit4] 2> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
[junit4] 2> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
[junit4] 2> at org.apache.solr.handler.CdcrRequestHandler.lambda$handleBootstrapAction$0(CdcrRequestHandler.java:646)
[junit4] 2> at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: java.lang.AssertionError
[junit4] 2> at org.apache.solr.handler.CdcrRequestHandler$BootstrapCallable.call(CdcrRequestHandler.java:789)
[junit4] 2> at org.apache.solr.handler.CdcrRequestHandler$BootstrapCallable.call(CdcrRequestHandler.java:716)
[junit4] 2> at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
[junit4] 2> ... 5 more
[junit4] 2> 348925 WARN (zkCallback-193-thread-3-processing-n:127.0.0.1:36989_solr) [n:127.0.0.1:36989_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 349140 ERROR (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[414FC989A1A4EA57]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 349329 WARN (Thread-120) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/aliases.json
[junit4] 2> 2 /solr/configs/cdcr-source
[junit4] 2> 2 /solr/security.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 13 /solr/collections/cdcr-source/state.json
[junit4] 2> 3 /solr/clusterstate.json
[junit4] 2> 3 /solr/clusterprops.json
[junit4] 2> 2 /solr/collections/cdcr-source/cdcr/state/process
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/collections
[junit4] 2> 2 /solr/overseer/queue
[junit4] 2> 2 /solr/live_nodes
[junit4] 2> 2 /solr/overseer/collection-queue-work
[junit4] 2> 2 /solr/overseer/queue-work
[junit4] 2>
[junit4] 2> 351051 WARN (zkCallback-168-thread-2-processing-n:127.0.0.1:34947_solr) [n:127.0.0.1:34947_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 351075 ERROR (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[414FC989A1A4EA57]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 351171 WARN (Thread-110) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/aliases.json
[junit4] 2> 2 /solr/configs/cdcr-target/managed-schema
[junit4] 2> 2 /solr/collections/cdcr-target/leaders/shard1/leader
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 6 /solr/collections/cdcr-target/state.json
[junit4] 2> 3 /solr/collections/cdcr-target/cdcr/state/process
[junit4] 2> 3 /solr/clusterstate.json
[junit4] 2> 3 /solr/clusterprops.json
[junit4] 2> 2 /solr/collections/cdcr-target/cdcr/state/buffer
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/live_nodes
[junit4] 2> 3 /solr/collections
[junit4] 2>
[junit4] 2> 351219 ERROR (Thread-153) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 351457 ERROR (jetty-launcher-202-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 1> Target zkHost = 127.0.0.1:44782/solr
[junit4] 2> 353008 ERROR (Thread-163) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 353123 ERROR (jetty-launcher-214-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 1> Adding 100 docs with commit=true, numDocs=100
[junit4] 1> Adding 100 docs with commit=true, numDocs=200
[junit4] 1> Adding 100 docs with commit=true, numDocs=300
[junit4] 1> Adding 100 docs with commit=true, numDocs=400
[junit4] 1> Adding 100 docs with commit=true, numDocs=500
[junit4] 1> Adding 100 docs with commit=true, numDocs=600
[junit4] 1> Adding 100 docs with commit=true, numDocs=700
[junit4] 1> Adding 100 docs with commit=true, numDocs=800
[junit4] 1> Adding 100 docs with commit=true, numDocs=900
[junit4] 1> Adding 100 docs with commit=true, numDocs=1000
[junit4] 1> Adding 100 docs with commit=true, numDocs=1100
[junit4] 1> Adding 100 docs with commit=true, numDocs=1200
[junit4] 1> Adding 100 docs with commit=true, numDocs=1300
[junit4] 1> Adding 100 docs with commit=true, numDocs=1400
[junit4] 1> Adding 100 docs with commit=true, numDocs=1500
[junit4] 1> Adding 100 docs with commit=true, numDocs=1600
[junit4] 1> Adding 100 docs with commit=true, numDocs=1700
[junit4] 1> Adding 100 docs with commit=true, numDocs=1800
[junit4] 1> Adding 100 docs with commit=true, numDocs=1900
[junit4] 1> Adding 100 docs with commit=true, numDocs=2000
[junit4] 1> Adding 100 docs with commit=true, numDocs=2100
[junit4] 1> Adding 100 docs with commit=true, numDocs=2200
[junit4] 1> Adding 100 docs with commit=true, numDocs=2300
[junit4] 1> Adding 100 docs with commit=true, numDocs=2400
[junit4] 1> Adding 100 docs with commit=true, numDocs=2500
[junit4] 1> Adding 100 docs with commit=true, numDocs=2600
[junit4] 1> Adding 100 docs with commit=true, numDocs=2700
[junit4] 1> Adding 100 docs with commit=true, numDocs=2800
[junit4] 1> Adding 100 docs with commit=true, numDocs=2900
[junit4] 1> Adding 100 docs with commit=true, numDocs=3000
[junit4] 1> Adding 100 docs with commit=true, numDocs=3100
[junit4] 1> Adding 100 docs with commit=true, numDocs=3200
[junit4] 1> Adding 100 docs with commit=true, numDocs=3300
[junit4] 1> Adding 100 docs with commit=true, numDocs=3400
[junit4] 1> Adding 100 docs with commit=true, numDocs=3500
[junit4] 1> Adding 100 docs with commit=true, numDocs=3600
[junit4] 1> Adding 100 docs with commit=true, numDocs=3700
[junit4] 1> Adding 100 docs with commit=true, numDocs=3800
[junit4] 1> Adding 100 docs with commit=true, numDocs=3900
[junit4] 1> Adding 100 docs with commit=true, numDocs=4000
[junit4] 1> Adding 100 docs with commit=true, numDocs=4100
[junit4] 1> Adding 100 docs with commit=true, numDocs=4200
[junit4] 1> Adding 100 docs with commit=true, numDocs=4300
[junit4] 1> Adding 100 docs with commit=true, numDocs=4400
[junit4] 1> Adding 100 docs with commit=true, numDocs=4500
[junit4] 1> Adding 100 docs with commit=true, numDocs=4600
[junit4] 1> Adding 100 docs with commit=true, numDocs=4700
[junit4] 1> Adding 100 docs with commit=true, numDocs=4800
[junit4] 1> Adding 100 docs with commit=true, numDocs=4900
[junit4] 1> Adding 100 docs with commit=true, numDocs=5000
[junit4] 1> Adding 100 docs with commit=true, numDocs=5100
[junit4] 1> Adding 100 docs with commit=true, numDocs=5200
[junit4] 1> Adding 100 docs with commit=true, numDocs=5300
[junit4] 1> Adding 100 docs with commit=true, numDocs=5400
[junit4] 1> Adding 100 docs with commit=true, numDocs=5500
[junit4] 1> Adding 100 docs with commit=true, numDocs=5600
[junit4] 1> Adding 100 docs with commit=true, numDocs=5700
[junit4] 1> Adding 100 docs with commit=true, numDocs=5800
[junit4] 1> Adding 100 docs with commit=true, numDocs=5900
[junit4] 1> Adding 100 docs with commit=true, numDocs=6000
[junit4] 1> Adding 100 docs with commit=true, numDocs=6100
[junit4] 1> Adding 100 docs with commit=true, numDocs=6200
[junit4] 1> Adding 100 docs with commit=true, numDocs=6300
[junit4] 1> Adding 100 docs with commit=true, numDocs=6400
[junit4] 1> Adding 100 docs with commit=true, numDocs=6500
[junit4] 1> Adding 100 docs with commit=true, numDocs=6600
[junit4] 1> Adding 100 docs with commit=true, numDocs=6700
[junit4] 1> Adding 100 docs with commit=true, numDocs=6800
[junit4] 1> Adding 100 docs with commit=true, numDocs=6900
[junit4] 1> Adding 100 docs with commit=true, numDocs=7000
[junit4] 1> Adding 100 docs with commit=true, numDocs=7100
[junit4] 1> Adding 100 docs with commit=true, numDocs=7200
[junit4] 1> Adding 100 docs with commit=true, numDocs=7300
[junit4] 1> Adding 100 docs with commit=true, numDocs=7400
[junit4] 1> Adding 100 docs with commit=true, numDocs=7500
[junit4] 1> Adding 100 docs with commit=true, numDocs=7600
[junit4] 1> Adding 100 docs with commit=true, numDocs=7700
[junit4] 1> Adding 100 docs with commit=true, numDocs=7800
[junit4] 1> Adding 100 docs with commit=true, numDocs=7900
[junit4] 1> Adding 100 docs with commit=true, numDocs=8000
[junit4] 1> Adding 100 docs with commit=true, numDocs=8100
[junit4] 1> Adding 100 docs with commit=true, numDocs=8200
[junit4] 1> Adding 100 docs with commit=true, numDocs=8300
[junit4] 1> Adding 100 docs with commit=true, numDocs=8400
[junit4] 1> Adding 100 docs with commit=true, numDocs=8500
[junit4] 1> Adding 100 docs with commit=true, numDocs=8600
[junit4] 1> Adding 100 docs with commit=true, numDocs=8700
[junit4] 1> Adding 100 docs with commit=true, numDocs=8800
[junit4] 1> Adding 100 docs with commit=true, numDocs=8900
[junit4] 1> Adding 100 docs with commit=true, numDocs=9000
[junit4] 1> Adding 100 docs with commit=true, numDocs=9100
[junit4] 1> Adding 100 docs with commit=true, numDocs=9200
[junit4] 1> Adding 100 docs with commit=true, numDocs=9300
[junit4] 1> Adding 100 docs with commit=true, numDocs=9400
[junit4] 1> Adding 100 docs with commit=true, numDocs=9500
[junit4] 1> Adding 100 docs with commit=true, numDocs=9600
[junit4] 1> Adding 100 docs with commit=true, numDocs=9700
[junit4] 1> Adding 100 docs with commit=true, numDocs=9800
[junit4] 1> Adding 100 docs with commit=true, numDocs=9900
[junit4] 1> Adding 100 docs with commit=true, numDocs=10000
[junit4] 1> Adding 100 docs with commit=true, numDocs=10100
[junit4] 2> 406330 WARN (cdcr-bootstrap-status-233-thread-1-processing-n:127.0.0.1:38859_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node2) [n:127.0.0.1:38859_solr c:cdcr-source s:shard1 r:core_node2 x:cdcr-source_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager Bootstrap process was not found on target collection: cdcr-target shard: shard1, leader: https://127.0.0.1:40181/solr/cdcr-target_shard1_replica_n1/
[junit4] 1> Adding 100 docs with commit=true, numDocs=10200
[junit4] 1> Adding 100 docs with commit=true, numDocs=10300
[junit4] 1> Adding 100 docs with commit=true, numDocs=10400
[junit4] 1> Adding 100 docs with commit=true, numDocs=10500
[junit4] 1> Adding 100 docs with commit=true, numDocs=10600
[junit4] 1> Adding 100 docs with commit=true, numDocs=10700
[junit4] 1> Adding 100 docs with commit=true, numDocs=10800
[junit4] 2> 411504 ERROR (updateExecutor-206-thread-1-processing-n:127.0.0.1:40181_solr x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node2) [n:127.0.0.1:40181_solr c:cdcr-target s:shard1 r:core_node2 x:cdcr-target_shard1_replica_n1] o.a.s.h.CdcrRequestHandler Bootstrap operation failed
[junit4] 2> java.util.concurrent.ExecutionException: java.lang.AssertionError
[junit4] 2> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
[junit4] 2> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
[junit4] 2> at org.apache.solr.handler.CdcrRequestHandler.lambda$handleBootstrapAction$0(CdcrRequestHandler.java:646)
[junit4] 2> at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: java.lang.AssertionError
[junit4] 2> at org.apache.solr.handler.CdcrRequestHandler$BootstrapCallable.call(CdcrRequestHandler.java:789)
[junit4] 2> at org.apache.solr.handler.CdcrRequestHandler$BootstrapCallable.call(CdcrRequestHandler.java:716)
[junit4] 2> at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
[junit4] 2> ... 5 more
[junit4] 1> Adding 100 docs with commit=true, numDocs=10900
[junit4] 1> Adding 100 docs with commit=true, numDocs=11000
[junit4] 1> Adding 100 docs with commit=true, numDocs=11100
[junit4] 1> Adding 100 docs with commit=true, numDocs=11200
[junit4] 1> Adding 100 docs with commit=true, numDocs=11300
[junit4] 1> Adding 100 docs with commit=true, numDocs=11400
[junit4] 1> Adding 100 docs with commit=true, numDocs=11500
[junit4] 1> Adding 100 docs with commit=true, numDocs=11600
[junit4] 1> Adding 100 docs with commit=true, numDocs=11700
[junit4] 1> Adding 100 docs with commit=true, numDocs=11800
[junit4] 1> Adding 100 docs with commit=true, numDocs=11900
[junit4] 1> Adding 100 docs with commit=true, numDocs=12000
[junit4] 1> Adding 100 docs with commit=true, numDocs=12100
[junit4] 1> Adding 100 docs with commit=true, numDocs=12200
[junit4] 1> Adding 100 docs with commit=true, numDocs=12300
[junit4] 1> Adding 100 docs with commit=true, numDocs=12400
[junit4] 1> Adding 100 docs with commit=true, numDocs=12500
[junit4] 1> Adding 100 docs with commit=true, numDocs=12600
[junit4] 1> Adding 100 docs with commit=true, numDocs=12700
[junit4] 1> Adding 100 docs with commit=true, numDocs=12800
[junit4] 1> Adding 100 docs with commit=true, numDocs=12900
[junit4] 1> Adding 100 docs with commit=true, numDocs=13000
[junit4] 1> Adding 100 docs with commit=true, numDocs=13100
[junit4] 1> Adding 100 docs with commit=true, numDocs=13200
[junit4] 1> Adding 100 docs with commit=true, numDocs=13300
[junit4] 1> Adding 100 docs with commit=true, numDocs=13400
[junit4] 1> Adding 100 docs with commit=true, numDocs=13500
[junit4] 1> Adding 100 docs with commit=true, numDocs=13600
[junit4] 1> Adding 100 docs with commit=true, numDocs=13700
[junit4] 1> Adding 100 docs with commit=true, numDocs=13800
[junit4] 1> Adding 100 docs with commit=true, numDocs=13900
[junit4] 1> Adding 100 docs with commit=true, numDocs=14000
[junit4] 1> Adding 100 docs with commit=true, numDocs=14100
[junit4] 1> Adding 100 docs with commit=true, numDocs=14200
[junit4] 1> Adding 100 docs with commit=true, numDocs=14300
[junit4] 1> Adding 100 docs with commit=true, numDocs=14400
[junit4] 1> Adding 100 docs with commit=true, numDocs=14500
[junit4] 1> Adding 100 docs with commit=true, numDocs=14600
[junit4] 1> Adding 100 docs with commit=true, numDocs=14700
[junit4]
[...truncated too long message...]
andler @ source_collection:shard1
[junit4] 2> 4170425 INFO (zkCallback-837-thread-1-processing-n:127.0.0.1:34461_) [n:127.0.0.1:34461_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 4170481 INFO (zkCallback-837-thread-1-processing-n:127.0.0.1:34461_) [n:127.0.0.1:34461_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/source_collection/state.json] for collection [source_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 4171925 WARN (zkCallback-898-thread-1-processing-n:127.0.0.1:36483_) [n:127.0.0.1:36483_ ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 4171926 INFO (TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[414FC989A1A4EA57]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@5251415a{HTTP/1.1,[http/1.1]}{127.0.0.1:36483}
[junit4] 2> 4171926 INFO (TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[414FC989A1A4EA57]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@59251a6d{/,null,UNAVAILABLE}
[junit4] 2> 4171927 INFO (TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[414FC989A1A4EA57]) [ ] o.a.s.c.ChaosMonkey monkey: stop jetty! 34461
[junit4] 2> 4171927 INFO (TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[414FC989A1A4EA57]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=2023714256
[junit4] 2> 4171927 INFO (TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[414FC989A1A4EA57]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 4171927 INFO (TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[414FC989A1A4EA57]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6179532d: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@2feb5fc5
[junit4] 2> 4171944 INFO (TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[414FC989A1A4EA57]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 4171944 INFO (TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[414FC989A1A4EA57]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@29125df7: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@5c00bd90
[junit4] 2> 4171945 INFO (TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[414FC989A1A4EA57]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 4171945 INFO (TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[414FC989A1A4EA57]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@796462c2: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@2c259142
[junit4] 2> 4171946 INFO (TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[414FC989A1A4EA57]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 4171948 INFO (coreCloseExecutor-1866-thread-1) [n:127.0.0.1:34461_ c:source_collection s:shard1 r:core_node3 x:source_collection_shard1_replica_n1] o.a.s.c.SolrCore [source_collection_shard1_replica_n1] CLOSING SolrCore org.apache.solr.core.SolrCore@e205d92
[junit4] 2> 4171948 INFO (coreCloseExecutor-1866-thread-1) [n:127.0.0.1:34461_ c:source_collection s:shard1 r:core_node3 x:source_collection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.source_collection.shard1.replica_n1, tag=237002130
[junit4] 2> 4171948 INFO (coreCloseExecutor-1866-thread-1) [n:127.0.0.1:34461_ c:source_collection s:shard1 r:core_node3 x:source_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4a3a2b40: rootName = null, domain = solr.core.source_collection.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.source_collection.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@611a09a4
[junit4] 2> 4171982 INFO (coreCloseExecutor-1866-thread-1) [n:127.0.0.1:34461_ c:source_collection s:shard1 r:core_node3 x:source_collection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.source_collection.shard1.leader, tag=237002130
[junit4] 2> 4171982 INFO (coreCloseExecutor-1866-thread-1) [n:127.0.0.1:34461_ c:source_collection s:shard1 r:core_node3 x:source_collection_shard1_replica_n1] o.a.s.h.CdcrRequestHandler Solr core is being closed - shutting down CDCR handler @ source_collection:shard1
[junit4] 2> 4171984 INFO (TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[414FC989A1A4EA57]) [ ] o.a.s.c.Overseer Overseer (id=98565184670859271-127.0.0.1:34461_-n_0000000001) closing
[junit4] 2> 4171997 INFO (OverseerStateUpdate-98565184670859271-127.0.0.1:34461_-n_0000000001) [n:127.0.0.1:34461_ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:34461_
[junit4] 2> 4171999 WARN (zkCallback-837-thread-1-processing-n:127.0.0.1:34461_) [n:127.0.0.1:34461_ ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 4172000 INFO (TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[414FC989A1A4EA57]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@46bd95a1{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 4172000 INFO (TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[414FC989A1A4EA57]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@106d5432{/,null,UNAVAILABLE}
[junit4] 2> 4172000 INFO (TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[414FC989A1A4EA57]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testPartialReplication
[junit4] 2> 4172000 ERROR (TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[414FC989A1A4EA57]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 4172001 INFO (TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[414FC989A1A4EA57]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:42587 42587
[junit4] 2> 4172121 INFO (Thread-123659) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:42587 42587
[junit4] 2> 4172121 WARN (Thread-123659) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 22 /solr/aliases.json
[junit4] 2> 5 /solr/configs/conf1
[junit4] 2> 4 /solr/collections/source_collection/leaders/shard1/leader
[junit4] 2> 3 /solr/security.json
[junit4] 2> 2 /solr/collections/tmp_collection/leaders/shard1/leader
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 22 /solr/clusterprops.json
[junit4] 2> 22 /solr/clusterstate.json
[junit4] 2> 4 /solr/collections/source_collection/cdcr/state/buffer
[junit4] 2> 4 /solr/collections/source_collection/cdcr/state/process
[junit4] 2> 3 /solr/collections/source_collection/state.json
[junit4] 2> 2 /solr/collections/tmp_collection/state.json
[junit4] 2> 2 /solr/collections/source_collection/leader_elect/shard1/election/98565184670859271-core_node3-n_0000000000
[junit4] 2> 2 /solr/collections/tmp_collection/cdcr/state/buffer
[junit4] 2> 2 /solr/collections/tmp_collection/cdcr/state/process
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 22 /solr/collections
[junit4] 2> 21 /solr/live_nodes
[junit4] 2> 2 /solr/overseer/queue
[junit4] 2> 2 /solr/overseer/collection-queue-work
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationHandlerTest_414FC989A1A4EA57-001
[junit4] 2> Aug 29, 2017 5:43:03 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Lucene70, sim=RandomSimilarity(queryNorm=true): {}, locale=en, timezone=America/Argentina/Rio_Gallegos
[junit4] 2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=220602664,total=523763712
[junit4] 2> NOTE: All tests run in this JVM: [MergeStrategyTest, HdfsBasicDistributedZkTest, ZkStateReaderTest, JavabinLoaderTest, TestHashPartitioner, TestFiltering, ScriptEngineTest, CloneFieldUpdateProcessorFactoryTest, TestObjectReleaseTracker, TestSubQueryTransformer, AlternateDirectoryTest, SchemaVersionSpecificBehaviorTest, TestManagedSchema, TestDelegationWithHadoopAuth, DocValuesMultiTest, TestCoreContainer, TestSerializedLuceneMatchVersion, TestConfigSets, MoveReplicaHDFSTest, SimpleCollectionCreateDeleteTest, DisMaxRequestHandlerTest, CleanupOldIndexTest, ConnectionReuseTest, CollectionsAPIDistributedZkTest, BlockJoinFacetRandomTest, TestStandardQParsers, TestSchemaResource, BasicFunctionalityTest, TestCoreAdminApis, CdcrReplicationHandlerTest]
[junit4] Completed [639/731 (3!)] on J1 in 144.57s, 5 tests, 1 failure <<< FAILURES!
[...truncated 46471 lines...]
[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 34 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/34/
4 tests failed.
FAILED: org.apache.solr.cloud.hdfs.HdfsUnloadDistributedZkTest.test
Error Message:
Error from server at http://127.0.0.1:43384: ADDREPLICA failed to create replica
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:43384: ADDREPLICA failed to create replica
at __randomizedtesting.SeedInfo.seed([85F6F73912499F48:DA2C8E3BCB5F2B0]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:627)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1121)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:862)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:793)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195)
at org.apache.solr.cloud.UnloadDistributedZkTest.testUnloadShardAndCollection(UnloadDistributedZkTest.java:124)
at org.apache.solr.cloud.UnloadDistributedZkTest.test(UnloadDistributedZkTest.java:71)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster
Error Message:
Document mismatch on target after sync expected:<20000> but was:<10100>
Stack Trace:
java.lang.AssertionError: Document mismatch on target after sync expected:<20000> but was:<10100>
at __randomizedtesting.SeedInfo.seed([85F6F73912499F48:51B3BC60F51F2CB3]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.apache.solr.cloud.CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster(CdcrBootstrapTest.java:309)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.hdfs.HdfsRestartWhileUpdatingTest.test
Error Message:
Captured an uncaught exception in thread: Thread[id=51250, name=updateExecutor-5140-thread-4, state=RUNNABLE, group=TGRP-HdfsRestartWhileUpdatingTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=51250, name=updateExecutor-5140-thread-4, state=RUNNABLE, group=TGRP-HdfsRestartWhileUpdatingTest]
at __randomizedtesting.SeedInfo.seed([85F6F73912499F48:DA2C8E3BCB5F2B0]:0)
Caused by: org.apache.solr.common.SolrException: Replica: http://127.0.0.1:46767/ugu/collection1_shard1_replica_n23/ should have been marked under leader initiated recovery in ZkController but wasn't.
at __randomizedtesting.SeedInfo.seed([85F6F73912499F48]:0)
at org.apache.solr.cloud.LeaderInitiatedRecoveryThread.run(LeaderInitiatedRecoveryThread.java:88)
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test
Error Message:
expected:<2> but was:<3>
Stack Trace:
java.lang.AssertionError: expected:<2> but was:<3>
at __randomizedtesting.SeedInfo.seed([85F6F73912499F48:DA2C8E3BCB5F2B0]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.assertSliceAndReplicaCount(SharedFSAutoReplicaFailoverTest.java:403)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.testBasics(SharedFSAutoReplicaFailoverTest.java:248)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test(SharedFSAutoReplicaFailoverTest.java:136)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12076 lines...]
[junit4] Suite: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest
[junit4] 2> 610479 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[85F6F73912499F48]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_85F6F73912499F48-001/init-core-data-001
[junit4] 2> 610480 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[85F6F73912499F48]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=56 numCloses=56
[junit4] 2> 610480 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[85F6F73912499F48]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 610493 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[85F6F73912499F48]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=None)
[junit4] 2> 610493 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[85F6F73912499F48]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /xwz/e
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 610903 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[85F6F73912499F48]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 610981 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[85F6F73912499F48]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 610982 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[85F6F73912499F48]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 611107 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[85F6F73912499F48]-worker) [ ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/hdfs to ./temp/Jetty_lucene2.us.west_apache_org_45323_hdfs____.h3ndoj/webapp
[junit4] 2> 612400 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[85F6F73912499F48]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@lucene2-us-west.apache.org:45323
[junit4] 2> 613095 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[85F6F73912499F48]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 613125 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[85F6F73912499F48]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 613174 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[85F6F73912499F48]-worker) [ ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_36334_datanode____.tliadk/webapp
[junit4] 2> 613885 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[85F6F73912499F48]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36334
[junit4] 2> 614478 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[85F6F73912499F48]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 614638 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[85F6F73912499F48]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 614711 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[85F6F73912499F48]-worker) [ ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_45802_datanode____im3hxc/webapp
[junit4] 2> 615091 ERROR (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_85F6F73912499F48-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_85F6F73912499F48-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:33303) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 615258 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0xd751419b4a2bb: from storage DS-7966d5e5-7894-4d4b-bf44-48221423afd0 node DatanodeRegistration(127.0.0.1:35068, datanodeUuid=f9ef26f9-fd50-4f27-9e46-521f8ef6ec1e, infoPort=45809, infoSecurePort=0, ipcPort=40963, storageInfo=lv=-56;cid=testClusterID;nsid=1261581380;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 615258 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0xd751419b4a2bb: from storage DS-aa78e473-fc9e-49a2-a90a-1d046a12f5eb node DatanodeRegistration(127.0.0.1:35068, datanodeUuid=f9ef26f9-fd50-4f27-9e46-521f8ef6ec1e, infoPort=45809, infoSecurePort=0, ipcPort=40963, storageInfo=lv=-56;cid=testClusterID;nsid=1261581380;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 615760 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[85F6F73912499F48]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45802
[junit4] 2> 616712 ERROR (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_85F6F73912499F48-001/tempDir-001/hdfsBaseDir/data/data3/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_85F6F73912499F48-001/tempDir-001/hdfsBaseDir/data/data4/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:33303) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 617045 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0xd7514843a4a7d: from storage DS-2580dea6-1485-48fd-ba1a-3a70386b3293 node DatanodeRegistration(127.0.0.1:39460, datanodeUuid=cf1cb528-d6f2-45de-8130-95b4b3699965, infoPort=33406, infoSecurePort=0, ipcPort=36511, storageInfo=lv=-56;cid=testClusterID;nsid=1261581380;c=0), blocks: 0, hasStaleStorage: true, processing time: 1 msecs
[junit4] 2> 617045 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0xd7514843a4a7d: from storage DS-abcd34a0-3f8b-4021-8830-ec2f67708429 node DatanodeRegistration(127.0.0.1:39460, datanodeUuid=cf1cb528-d6f2-45de-8130-95b4b3699965, infoPort=33406, infoSecurePort=0, ipcPort=36511, storageInfo=lv=-56;cid=testClusterID;nsid=1261581380;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 617225 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 617238 INFO (Thread-640) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 617239 INFO (Thread-640) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 617271 ERROR (Thread-640) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 617340 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.c.ZkTestServer start zk server on port:34613
[junit4] 2> 617828 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 617830 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 617831 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 617846 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 617847 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 617860 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 617861 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 617861 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 617862 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 617863 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 617863 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 617881 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e1a3a85ef0001, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 617881 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
[junit4] 2> 618416 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 618429 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@62c501cf{/xwz/e,null,AVAILABLE}
[junit4] 2> 618429 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@537a1afb{HTTP/1.1,[http/1.1]}{127.0.0.1:34990}
[junit4] 2> 618429 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.e.j.s.Server Started @631283ms
[junit4] 2> 618429 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/xwz/e, hostPort=34990, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_85F6F73912499F48-001/control-001/cores}
[junit4] 2> 618430 ERROR (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 618430 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
[junit4] 2> 618430 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 618430 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 618430 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-25T16:29:57.782Z
[junit4] 2> 618534 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 618534 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_85F6F73912499F48-001/control-001/solr.xml
[junit4] 2> 618552 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@990b9fa, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 618583 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34613/solr
[junit4] 2> 619527 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [n:127.0.0.1:34990_xwz%2Fe ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 619527 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [n:127.0.0.1:34990_xwz%2Fe ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:34990_xwz%2Fe
[junit4] 2> 619527 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [n:127.0.0.1:34990_xwz%2Fe ] o.a.s.c.Overseer Overseer (id=98545080506187780-127.0.0.1:34990_xwz%2Fe-n_0000000000) starting
[junit4] 2> 619634 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [n:127.0.0.1:34990_xwz%2Fe ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34990_xwz%2Fe
[junit4] 2> 619707 INFO (zkCallback-247-thread-1-processing-n:127.0.0.1:34990_xwz%2Fe) [n:127.0.0.1:34990_xwz%2Fe ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 620287 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [n:127.0.0.1:34990_xwz%2Fe ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@990b9fa
[junit4] 2> 620319 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [n:127.0.0.1:34990_xwz%2Fe ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@990b9fa
[junit4] 2> 620319 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [n:127.0.0.1:34990_xwz%2Fe ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@990b9fa
[junit4] 2> 620333 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [n:127.0.0.1:34990_xwz%2Fe ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_85F6F73912499F48-001/control-001/cores
[junit4] 2> 620512 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 620512 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34613/solr ready
[junit4] 2> 620546 INFO (qtp233356006-1857) [n:127.0.0.1:34990_xwz%2Fe ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:34990_xwz%252Fe&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 620561 INFO (OverseerThreadFactory-899-thread-1-processing-n:127.0.0.1:34990_xwz%2Fe) [n:127.0.0.1:34990_xwz%2Fe ] o.a.s.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 620686 INFO (qtp233356006-1853) [n:127.0.0.1:34990_xwz%2Fe ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 620686 INFO (qtp233356006-1853) [n:127.0.0.1:34990_xwz%2Fe ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 620813 INFO (zkCallback-247-thread-2-processing-n:127.0.0.1:34990_xwz%2Fe) [n:127.0.0.1:34990_xwz%2Fe ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 621712 INFO (qtp233356006-1853) [n:127.0.0.1:34990_xwz%2Fe c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 621747 INFO (qtp233356006-1853) [n:127.0.0.1:34990_xwz%2Fe c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
[junit4] 2> 623950 INFO (qtp233356006-1853) [n:127.0.0.1:34990_xwz%2Fe c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 624095 INFO (qtp233356006-1853) [n:127.0.0.1:34990_xwz%2Fe c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
[junit4] 2> 624095 INFO (qtp233356006-1853) [n:127.0.0.1:34990_xwz%2Fe c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@990b9fa
[junit4] 2> 624096 INFO (qtp233356006-1853) [n:127.0.0.1:34990_xwz%2Fe c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://lucene2-us-west.apache.org:33303/solr_hdfs_home
[junit4] 2> 624096 INFO (qtp233356006-1853) [n:127.0.0.1:34990_xwz%2Fe c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 624096 INFO (qtp233356006-1853) [n:127.0.0.1:34990_xwz%2Fe c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 624096 INFO (qtp233356006-1853) [n:127.0.0.1:34990_xwz%2Fe c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_85F6F73912499F48-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[hdfs://lucene2-us-west.apache.org:33303/solr_hdfs_home/control_collection/core_node2/data/]
[junit4] 2> 624097 INFO (qtp233356006-1853) [n:127.0.0.1:34990_xwz%2Fe c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:33303/solr_hdfs_home/control_collection/core_node2/data/snapshot_metadata
[junit4] 2> 624150 INFO (qtp233356006-1853) [n:127.0.0.1:34990_xwz%2Fe c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 624150 INFO (qtp233356006-1853) [n:127.0.0.1:34990_xwz%2Fe c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
[junit4] 2> 624150 INFO (qtp233356006-1853) [n:127.0.0.1:34990_xwz%2Fe c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Creating new global HDFS BlockCache
[junit4] 2> 627319 INFO (qtp233356006-1853) [n:127.0.0.1:34990_xwz%2Fe c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 627320 INFO (qtp233356006-1853) [n:127.0.0.1:34990_xwz%2Fe c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:33303/solr_hdfs_home/control_collection/core_node2/data
[junit4] 2> 627595 INFO (qtp233356006-1853) [n:127.0.0.1:34990_xwz%2Fe c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:33303/solr_hdfs_home/control_collection/core_node2/data/index
[junit4] 2> 627644 INFO (qtp233356006-1853) [n:127.0.0.1:34990_xwz%2Fe c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 627644 INFO (qtp233356006-1853) [n:127.0.0.1:34990_xwz%2Fe c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
[junit4] 2> 627761 INFO (qtp233356006-1853) [n:127.0.0.1:34990_xwz%2Fe c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 627762 INFO (qtp233356006-1853) [n:127.0.0.1:34990_xwz%2Fe c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=29, maxMergeAtOnceExplicit=43, maxMergedSegmentMB=1.453125, floorSegmentMB=0.958984375, forceMergeDeletesPctAllowed=28.590470189983037, segmentsPerTier=42.0, maxCFSSegmentSizeMB=2.1298828125, noCFSRatio=1.0
[junit4] 2> 628498 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:35068 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-abcd34a0-3f8b-4021-8830-ec2f67708429:NORMAL:127.0.0.1:39460|RBW], ReplicaUC[[DISK]DS-aa78e473-fc9e-49a2-a90a-1d046a12f5eb:NORMAL:127.0.0.1:35068|FINALIZED]]} size 0
[junit4] 2> 628500 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:39460 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-abcd34a0-3f8b-4021-8830-ec2f67708429:NORMAL:127.0.0.1:39460|RBW], ReplicaUC[[DISK]DS-aa78e473-fc9e-49a2-a90a-1d046a12f5eb:NORMAL:127.0.0.1:35068|FINALIZED]]} size 0
[junit4] 2> 628656 WARN (qtp233356006-1853) [n:127.0.0.1:34990_xwz%2Fe c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 628848 INFO (qtp233356006-1853) [n:127.0.0.1:34990_xwz%2Fe c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 628848 INFO (qtp233356006-1853) [n:127.0.0.1:34990_xwz%2Fe c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 628848 INFO (qtp233356006-1853) [n:127.0.0.1:34990_xwz%2Fe c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 628927 INFO (qtp233356006-1853) [n:127.0.0.1:34990_xwz%2Fe c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 628927 INFO (qtp233356006-1853) [n:127.0.0.1:34990_xwz%2Fe c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 628956 INFO (qtp233356006-1853) [n:127.0.0.1:34990_xwz%2Fe c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=48, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6763145298064487]
[junit4] 2> 629185 INFO (qtp233356006-1853) [n:127.0.0.1:34990_xwz%2Fe c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3047acd4[control_collection_shard1_replica_n1] main]
[junit4] 2> 629195 INFO (qtp233356006-1853) [n:127.0.0.1:34990_xwz%2Fe c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 629195 INFO (qtp233356006-1853) [n:127.0.0.1:34990_xwz%2Fe c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 629198 INFO (qtp233356006-1853) [n:127.0.0.1:34990_xwz%2Fe c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 629199 INFO (qtp233356006-1853) [n:127.0.0.1:34990_xwz%2Fe c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1576721300639973376
[junit4] 2> 629247 INFO (qtp233356006-1853) [n:127.0.0.1:34990_xwz%2Fe c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 629247 INFO (qtp233356006-1853) [n:127.0.0.1:34990_xwz%2Fe c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 629247 INFO (qtp233356006-1853) [n:127.0.0.1:34990_xwz%2Fe c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:34990/xwz/e/control_collection_shard1_replica_n1/
[junit4] 2> 629260 INFO (zkCallback-247-thread-2-processing-n:127.0.0.1:34990_xwz%2Fe) [n:127.0.0.1:34990_xwz%2Fe ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 629270 INFO (searcherExecutor-902-thread-1-processing-n:127.0.0.1:34990_xwz%2Fe x:control_collection_shard1_replica_n1 s:shard1 c:control_collection) [n:127.0.0.1:34990_xwz%2Fe c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@3047acd4[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 629275 INFO (qtp233356006-1853) [n:127.0.0.1:34990_xwz%2Fe c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 629275 INFO (qtp233356006-1853) [n:127.0.0.1:34990_xwz%2Fe c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:34990/xwz/e/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 629275 INFO (qtp233356006-1853) [n:127.0.0.1:34990_xwz%2Fe c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 629296 INFO (qtp233356006-1853) [n:127.0.0.1:34990_xwz%2Fe c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:34990/xwz/e/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 629403 INFO (zkCallback-247-thread-2-processing-n:127.0.0.1:34990_xwz%2Fe) [n:127.0.0.1:34990_xwz%2Fe ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 629409 INFO (qtp233356006-1853) [n:127.0.0.1:34990_xwz%2Fe c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 629410 INFO (qtp233356006-1853) [n:127.0.0.1:34990_xwz%2Fe c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=8724
[junit4] 2> 629564 INFO (qtp233356006-1857) [n:127.0.0.1:34990_xwz%2Fe ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 629596 INFO (zkCallback-247-thread-2-processing-n:127.0.0.1:34990_xwz%2Fe) [n:127.0.0.1:34990_xwz%2Fe ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 629668 INFO (OverseerCollectionConfigSetProcessor-98545080506187780-127.0.0.1:34990_xwz%2Fe-n_0000000000) [n:127.0.0.1:34990_xwz%2Fe ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 630570 INFO (qtp233356006-1857) [n:127.0.0.1:34990_xwz%2Fe ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:34990_xwz%252Fe&wt=javabin&version=2} status=0 QTime=10023
[junit4] 2> 630636 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 630636 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34613/solr ready
[junit4] 2> 630636 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 630638 INFO (qtp233356006-1854) [n:127.0.0.1:34990_xwz%2Fe ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 630683 INFO (OverseerThreadFactory-899-thread-2-processing-n:127.0.0.1:34990_xwz%2Fe) [n:127.0.0.1:34990_xwz%2Fe ] o.a.s.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 630684 WARN (OverseerThreadFactory-899-thread-2-processing-n:127.0.0.1:34990_xwz%2Fe) [n:127.0.0.1:34990_xwz%2Fe ] o.a.s.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
[junit4] 2> 630928 INFO (qtp233356006-1854) [n:127.0.0.1:34990_xwz%2Fe ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 630928 INFO (qtp233356006-1854) [n:127.0.0.1:34990_xwz%2Fe ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=2&wt=javabin&version=2} status=0 QTime=290
[junit4] 2> 631496 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_85F6F73912499F48-001/shard-1-001 of type NRT
[junit4] 2> 631497 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 631572 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@23bfdcff{/xwz/e,null,AVAILABLE}
[junit4] 2> 631572 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3d8fa643{HTTP/1.1,[http/1.1]}{127.0.0.1:42349}
[junit4] 2> 631572 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.e.j.s.Server Started @644425ms
[junit4] 2> 631572 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml, hostContext=/xwz/e, hostPort=42349, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_85F6F73912499F48-001/shard-1-001/cores}
[junit4] 2> 631572 ERROR (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 631573 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
[junit4] 2> 631573 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 631573 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 631573 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-25T16:30:10.925Z
[junit4] 2> 631685 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 631685 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_85F6F73912499F48-001/shard-1-001/solr.xml
[junit4] 2> 631702 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@990b9fa, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 631734 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34613/solr
[junit4] 2> 631931 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [n:127.0.0.1:42349_xwz%2Fe ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 631933 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [n:127.0.0.1:42349_xwz%2Fe ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 631934 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [n:127.0.0.1:42349_xwz%2Fe ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42349_xwz%2Fe
[junit4] 2> 631937 INFO (zkCallback-259-thread-1-processing-n:127.0.0.1:42349_xwz%2Fe) [n:127.0.0.1:42349_xwz%2Fe ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 631958 INFO (zkCallback-247-thread-1-processing-n:127.0.0.1:34990_xwz%2Fe) [n:127.0.0.1:34990_xwz%2Fe ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 632094 INFO (zkCallback-254-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 632276 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [n:127.0.0.1:42349_xwz%2Fe ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@990b9fa
[junit4] 2> 632345 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [n:127.0.0.1:42349_xwz%2Fe ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@990b9fa
[junit4] 2> 632358 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [n:127.0.0.1:42349_xwz%2Fe ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@990b9fa
[junit4] 2> 632359 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [n:127.0.0.1:42349_xwz%2Fe ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_85F6F73912499F48-001/shard-1-001/cores
[junit4] 2> 632490 INFO (qtp1324605805-1908) [n:127.0.0.1:42349_xwz%2Fe ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:42349_xwz%252Fe&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 632520 INFO (OverseerCollectionConfigSetProcessor-98545080506187780-127.0.0.1:34990_xwz%2Fe-n_0000000000) [n:127.0.0.1:34990_xwz%2Fe ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 632522 INFO (OverseerThreadFactory-899-thread-3-processing-n:127.0.0.1:34990_xwz%2Fe) [n:127.0.0.1:34990_xwz%2Fe ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:42349_xwz%2Fe for creating new replica
[junit4] 2> 632563 INFO (qtp1324605805-1905) [n:127.0.0.1:42349_xwz%2Fe ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 632563 INFO (qtp1324605805-1905) [n:127.0.0.1:42349_xwz%2Fe ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 632702 INFO (zkCallback-259-thread-1-processing-n:127.0.0.1:42349_xwz%2Fe) [n:127.0.0.1:42349_xwz%2Fe ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 633675 INFO (qtp1324605805-1905) [n:127.0.0.1:42349_xwz%2Fe c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 633768 INFO (qtp1324605805-1905) [n:127.0.0.1:42349_xwz%2Fe c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.s.IndexSchema [collection1_shard2_replica_n41] Schema name=test
[junit4] 2> 634258 INFO (qtp1324605805-1905) [n:127.0.0.1:42349_xwz%2Fe c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 634340 INFO (qtp1324605805-1905) [n:127.0.0.1:42349_xwz%2Fe c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n41' using configuration from collection collection1, trusted=true
[junit4] 2> 634340 INFO (qtp1324605805-1905) [n:127.0.0.1:42349_xwz%2Fe c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_n41' (registry 'solr.core.collection1.shard2.replica_n41') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@990b9fa
[junit4] 2> 634340 INFO (qtp1324605805-1905) [n:127.0.0.1:42349_xwz%2Fe c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://lucene2-us-west.apache.org:33303/solr_hdfs_home
[junit4] 2> 634340 INFO (qtp1324605805-1905) [n:127.0.0.1:42349_xwz%2Fe c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 634340 INFO (qtp1324605805-1905) [n:127.0.0.1:42349_xwz%2Fe c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 634340 INFO (qtp1324605805-1905) [n:127.0.0.1:42349_xwz%2Fe c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.SolrCore [[collection1_shard2_replica_n41] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_85F6F73912499F48-001/shard-1-001/cores/collection1_shard2_replica_n41], dataDir=[hdfs://lucene2-us-west.apache.org:33303/solr_hdfs_home/collection1/core_node42/data/]
[junit4] 2> 634342 INFO (qtp1324605805-1905) [n:127.0.0.1:42349_xwz%2Fe c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:33303/solr_hdfs_home/collection1/core_node42/data/snapshot_metadata
[junit4] 2> 634439 INFO (qtp1324605805-1905) [n:127.0.0.1:42349_xwz%2Fe c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 634439 INFO (qtp1324605805-1905) [n:127.0.0.1:42349_xwz%2Fe c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
[junit4] 2> 634475 INFO (qtp1324605805-1905) [n:127.0.0.1:42349_xwz%2Fe c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 634489 INFO (qtp1324605805-1905) [n:127.0.0.1:42349_xwz%2Fe c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:33303/solr_hdfs_home/collection1/core_node42/data
[junit4] 2> 634669 INFO (qtp1324605805-1905) [n:127.0.0.1:42349_xwz%2Fe c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:33303/solr_hdfs_home/collection1/core_node42/data/index
[junit4] 2> 634750 INFO (qtp1324605805-1905) [n:127.0.0.1:42349_xwz%2Fe c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 634750 INFO (qtp1324605805-1905) [n:127.0.0.1:42349_xwz%2Fe c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
[junit4] 2> 634896 INFO (qtp1324605805-1905) [n:127.0.0.1:42349_xwz%2Fe c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 634896 INFO (qtp1324605805-1905) [n:127.0.0.1:42349_xwz%2Fe c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=29, maxMergeAtOnceExplicit=43, maxMergedSegmentMB=1.453125, floorSegmentMB=0.958984375, forceMergeDeletesPctAllowed=28.590470189983037, segmentsPerTier=42.0, maxCFSSegmentSizeMB=2.1298828125, noCFSRatio=1.0
[junit4] 2> 635452 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:39460 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-aa78e473-fc9e-49a2-a90a-1d046a12f5eb:NORMAL:127.0.0.1:35068|RBW], ReplicaUC[[DISK]DS-2580dea6-1485-48fd-ba1a-3a70386b3293:NORMAL:127.0.0.1:39460|FINALIZED]]} size 0
[junit4] 2> 635467 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:35068 is added to blk_1073741826_1002 size 72
[junit4] 2> 635546 WARN (qtp1324605805-1905) [n:127.0.0.1:42349_xwz%2Fe c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 635941 INFO (qtp1324605805-1905) [n:127.0.0.1:42349_xwz%2Fe c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 635941 INFO (qtp1324605805-1905) [n:127.0.0.1:42349_xwz%2Fe c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 635941 INFO (qtp1324605805-1905) [n:127.0.0.1:42349_xwz%2Fe c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 636122 INFO (qtp1324605805-1905) [n:127.0.0.1:42349_xwz%2Fe c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 636122 INFO (qtp1324605805-1905) [n:127.0.0.1:42349_xwz%2Fe c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 636154 INFO (qtp1324605805-1905) [n:127.0.0.1:42349_xwz%2Fe c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=48, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6763145298064487]
[junit4] 2> 636365 INFO (qtp1324605805-1905) [n:127.0.0.1:42349_xwz%2Fe c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.s.SolrIndexSearcher Opening [Searcher@ebba6ac[collection1_shard2_replica_n41] main]
[junit4] 2> 636381 INFO (qtp1324605805-1905) [n:127.0.0.1:42349_xwz%2Fe c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 636382 INFO (qtp1324605805-1905) [n:127.0.0.1:42349_xwz%2Fe c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 636382 INFO (qtp1324605805-1905) [n:127.0.0.1:42349_xwz%2Fe c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 636384 INFO (searcherExecutor-913-thread-1-processing-n:127.0.0.1:42349_xwz%2Fe x:collection1_shard2_replica_n41 s:shard2 c:collection1) [n:127.0.0.1:42349_xwz%2Fe c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.SolrCore [collection1_shard2_replica_n41] Registered new searcher Searcher@ebba6ac[collection1_shard2_replica_n41] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 636384 INFO (qtp1324605805-1905) [n:127.0.0.1:42349_xwz%2Fe c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1576721308173991936
[junit4] 2> 636495 INFO (qtp1324605805-1905) [n:127.0.0.1:42349_xwz%2Fe c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 636495 INFO (qtp1324605805-1905) [n:127.0.0.1:42349_xwz%2Fe c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 636495 INFO (qtp1324605805-1905) [n:127.0.0.1:42349_xwz%2Fe c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:42349/xwz/e/collection1_shard2_replica_n41/
[junit4] 2> 636495 INFO (qtp1324605805-1905) [n:127.0.0.1:42349_xwz%2Fe c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 636495 INFO (qtp1324605805-1905) [n:127.0.0.1:42349_xwz%2Fe c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.SyncStrategy http://127.0.0.1:42349/xwz/e/collection1_shard2_replica_n41/ has no replicas
[junit4] 2> 636495 INFO (qtp1324605805-1905) [n:127.0.0.1:42349_xwz%2Fe c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 636497 INFO (qtp1324605805-1905) [n:127.0.0.1:42349_xwz%2Fe c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:42349/xwz/e/collection1_shard2_replica_n41/ shard2
[junit4] 2> 636512 INFO (zkCallback-259-thread-1-processing-n:127.0.0.1:42349_xwz%2Fe) [n:127.0.0.1:42349_xwz%2Fe ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 636615 INFO (zkCallback-259-thread-1-processing-n:127.0.0.1:42349_xwz%2Fe) [n:127.0.0.1:42349_xwz%2Fe ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 636662 INFO (qtp1324605805-1905) [n:127.0.0.1:42349_xwz%2Fe c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 636664 INFO (qtp1324605805-1905) [n:127.0.0.1:42349_xwz%2Fe c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=4101
[junit4] 2> 636716 INFO (qtp1324605805-1908) [n:127.0.0.1:42349_xwz%2Fe ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:42349_xwz%252Fe&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2} status=0 QTime=4225
[junit4] 2> 636775 INFO (zkCallback-259-thread-1-processing-n:127.0.0.1:42349_xwz%2Fe) [n:127.0.0.1:42349_xwz%2Fe ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 638078 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_85F6F73912499F48-001/shard-2-001 of type NRT
[junit4] 2> 638079 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 638095 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@446200b2{/xwz/e,null,AVAILABLE}
[junit4] 2> 638096 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@482dddd1{HTTP/1.1,[http/1.1]}{127.0.0.1:42468}
[junit4] 2> 638096 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.e.j.s.Server Started @650949ms
[junit4] 2> 638096 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solrconfig=solrconfig.xml, hostContext=/xwz/e, hostPort=42468, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_85F6F73912499F48-001/shard-2-001/cores}
[junit4] 2> 638096 ERROR (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 638097 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
[junit4] 2> 638097 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 638097 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 638097 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-25T16:30:17.449Z
[junit4] 2> 638247 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 638247 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_85F6F73912499F48-001/shard-2-001/solr.xml
[junit4] 2> 638440 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@990b9fa, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 638475 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34613/solr
[junit4] 2> 638549 INFO (OverseerCollectionConfigSetProcessor-98545080506187780-127.0.0.1:34990_xwz%2Fe-n_0000000000) [n:127.0.0.1:34990_xwz%2Fe ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 638800 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [n:127.0.0.1:42468_xwz%2Fe ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 638830 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [n:127.0.0.1:42468_xwz%2Fe ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 638832 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [n:127.0.0.1:42468_xwz%2Fe ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42468_xwz%2Fe
[junit4] 2> 638832 INFO (zkCallback-259-thread-1-processing-n:127.0.0.1:42349_xwz%2Fe) [n:127.0.0.1:42349_xwz%2Fe ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 638833 INFO (zkCallback-247-thread-2-processing-n:127.0.0.1:34990_xwz%2Fe) [n:127.0.0.1:34990_xwz%2Fe ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 638850 INFO (zkCallback-254-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 638862 INFO (zkCallback-265-thread-1-processing-n:127.0.0.1:42468_xwz%2Fe) [n:127.0.0.1:42468_xwz%2Fe ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 639335 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [n:127.0.0.1:42468_xwz%2Fe ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@990b9fa
[junit4] 2> 639589 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [n:127.0.0.1:42468_xwz%2Fe ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@990b9fa
[junit4] 2> 639589 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [n:127.0.0.1:42468_xwz%2Fe ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@990b9fa
[junit4] 2> 639590 INFO (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[85F6F73912499F48]) [n:127.0.0.1:42468_xwz%2Fe ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_85F6F73912499F48-001/shard-2-001/cores
[junit4] 2> 639803 INFO (qtp1324605805-1909) [n:127.0.0.1:42349_xwz%2Fe ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:42468_xwz%252Fe&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 639820 INFO (OverseerThreadFactory-899-thread-4-processing-n:127.0.0.1:34990_xwz%2Fe) [n:127.0.0.1:34990_xwz%2Fe ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:42468_xwz%2Fe for creating new replica
[junit4] 2> 639864 INFO (qtp392675976-1948) [n:127.0.0.1:42468_xwz%2Fe ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n43&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 639864 INFO (qtp392675976-1948) [n:127.0.0.1:42468_xwz%2Fe ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 639919 INFO (zkCallback-259-thread-1-processing-n:127.0.0.1:42349_xwz%2Fe) [n:127.0.0.1:42349_xwz%2Fe ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 639968 INFO (qtp392675976-1948) [n:127.0.0.1:42468_xwz%2Fe c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 640033 INFO (qtp392675976-1948) [n:127.0.0.1:42468_xwz%2Fe c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.s.IndexSchema [collection1_shard1_replica_n43] Schema name=test
[junit4] 2> 640621 INFO (qtp392675976-1948) [n:127.0.0.1:42468_xwz%2Fe c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 640671 INFO (qtp392675976-1948) [n:127.0.0.1:42468_xwz%2Fe c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n43' using configuration from collection collection1, trusted=true
[junit4] 2> 640684 INFO (qtp392675976-1948) [n:127.0.0.1:42468_xwz%2Fe c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n43' (registry 'solr.core.collection1.shard1.replica_n43') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@990b9fa
[junit4] 2> 640684 INFO (qtp392675976-1948) [n:127.0.0.1:42468_xwz%2Fe c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://lucene2-us-west.apache.org:33303/solr_hdfs_home
[junit4] 2> 640684 INFO (qtp392675976-1948) [n:127.0.0.1:42468_xwz%2Fe c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 640684 INFO (qtp392675976-1948) [n:127.0.0.1:42468_xwz%2Fe c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 640701 INFO (qtp392675976-1948) [n:127.0.0.1:42468_xwz%2Fe c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.SolrCore [[collection1_shard1_replica_n43] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_85F6F73912499F48-001/shard-2-001/cores/collection1_shard1_replica_n43], dataDir=[hdfs://lucene2-us-west.apache.org:33303/solr_hdfs_home/collection1/core_node44/data/]
[junit4] 2> 640702 INFO (qtp392675976-1948) [n:127.0.0.1:42468_xwz%2Fe c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:33303/solr_hdfs_home/collection1/core_node44/data/snapshot_metadata
[junit4] 2> 640787 INFO (qtp392675976-1948) [n:127.0.0.1:42468_xwz%2Fe c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 640787 INFO (qtp392675976-1948) [n:127.0.0.1:42468_xwz%2Fe c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
[junit4] 2> 640835 INFO (qtp392675976-1948) [n:127.0.0.1:42468_xwz%2Fe c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 640862 INFO (qtp392675976-1948) [n:127.0.0.1:42468_xwz%2Fe c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:33303/solr_hdfs_home/collection1/core_node44/data
[junit4] 2> 641106 INFO (qtp392675976-1948) [n:127.0.0.1:42468_xwz%2Fe c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:33303/solr_hdfs_home/collection1/core_node44/data/index
[junit4] 2> 641125 INFO (qtp392675976-1948) [n:127.0.0.1:42468_xwz%2Fe c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 641125 INFO (qtp392675976-1948) [n:127.0.0.1:42468_xwz%2Fe c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
[junit4] 2> 641187 INFO (qtp392675976-1948) [n:127.0.0.1:42468_xwz%2Fe c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 641188 INFO (qtp392675976-1948) [n:127.0.0.1:42468_xwz%2Fe c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=29, maxMergeAtOnceExplicit=43, maxMergedSegmentMB=1.453125, floorSegmentMB=0.958984375, forceMergeDeletesPctAllowed=28.590470189983037, segmentsPerTier=42.0, maxCFSSegmentSizeMB=2.1298828125, noCFSRatio=1.0
[junit4] 2> 641565 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:35068 is added to blk_1073741827_1003{UCState=COMMITTED, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-abcd34a0-3f8b-4021-8830-ec2f67708429:NORMAL:127.0.0.1:39460|RBW], ReplicaUC[[DISK]DS-7966d5e5-7894-4d4b-bf44-48221423afd0:NORMAL:127.0.0.1:35068|RBW]]} size 72
[junit4] 2> 641581 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:39460 is added to blk_1073741827_1003 size 72
[junit4] 2> 642013 WARN (qtp392675976-1948) [n:127.0.0.1:42468_xwz%2Fe c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 642373 INFO (qtp392675976-1948) [n:127.0.0.1:42468_xwz%2Fe c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 642373 INFO (qtp392675976-1948) [n:127.0.0.1:42468_xwz%2Fe c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 642373 INFO (qtp392675976-1948) [n:127.0.0.1:42468_xwz%2Fe c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 642508 INFO (qtp392675976-1948) [n:127.0.0.1:42468_xwz%2Fe c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 642508 INFO (qtp392675976-1948) [n:127.0.0.1:42468_xwz%2Fe c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 642510 INFO (qtp392675976-1948) [n:127.0.0.1:42468_xwz%2Fe c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=48, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6763145298064487]
[junit4] 2> 642803 INFO (qtp392675976-1948) [n:127.0.0.1:42468_xwz%2Fe c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.s.SolrIndexSearcher Opening [Searcher@6e85d6df[collection1_shard1_replica_n43] main]
[junit4] 2> 642820 INFO (qtp392675976-1948) [n:127.0.0.1:42468_xwz%2Fe c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 642820 INFO (qtp392675976-1948) [n:127.0.0.1:42468_xwz%2Fe c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 642820 INFO (qtp392675976-1948) [n:127.0.0.1:42468_xwz%2Fe c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 642821 INFO (qtp392675976-1948) [n:127.0.0.1:42468_xwz%2Fe c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1576721314923675648
[junit4] 2> 642862 INFO (searcherExecutor-924-thread-1-processing-n:127.0.0.1:42468_xwz%2Fe x:collection1_shard1_replica_n43 s:shard1 c:collection1) [n:127.0.0.1:42468_xwz%2Fe c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.SolrCore [collection1_shard1_replica_n43] Registered new searcher Searcher@6e85d6df[collection1_shard1_replica_n43] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 642863 INFO (qtp392675976-1948) [n:127.0.0.1:42468_xwz%2Fe c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 642863 INFO (qtp392675976-1948) [n:127.0.0.1:42468_xwz%2Fe c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 642863 INFO (qtp392675976-1948) [n:127.0.0.1:42468_xwz%2Fe c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:42468/xwz/e/collection1_shard1_replica_n43/
[junit4] 2> 642863 INFO (qtp392675976-1948) [n:127.0.0.1:42468_xwz%2Fe c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 642863 INFO (qtp392675976-1948) [n:127.0.0.1:42468_xwz%2Fe c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.SyncStrategy http://127.0.0.1:42468/xwz/e/collection1_shard1_replica_n43/ has no replicas
[junit4] 2> 642863 INFO (qtp392675976-1948) [n:127.0.0.1:42468_xwz%2Fe c:collection1 s:shard1 x:collection1_shard1_replica_n43] o.a.s.c.ShardLeaderElec
[...truncated too long message...]
shutdown has been called
[junit4] 2> 9212527 WARN (1725321157@qtp-405133530-1 - Acceptor0 HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33654) [ ] o.a.h.h.HttpServer2 HttpServer Acceptor: isRunning is false. Rechecking.
[junit4] 2> 9212527 WARN (1725321157@qtp-405133530-1 - Acceptor0 HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33654) [ ] o.a.h.h.HttpServer2 HttpServer Acceptor: isRunning is false
[junit4] 2> 9212546 WARN (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_85F6F73912499F48-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_85F6F73912499F48-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:41966) [ ] o.a.h.h.s.d.IncrementalBlockReportManager IncrementalBlockReportManager interrupted
[junit4] 2> 9212547 WARN (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_85F6F73912499F48-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_85F6F73912499F48-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:41966) [ ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-222479059-127.0.0.1-1503687009571 (Datanode Uuid 7687219b-0ab5-46eb-b777-3d03235d88b5) service to lucene2-us-west.apache.org/127.0.0.1:41966
[junit4] 2> 9212592 WARN (1660392323@qtp-1808189085-1 - Acceptor0 HttpServer2$SelectChannelConnectorWithSafeStartup@lucene2-us-west.apache.org:33736) [ ] o.a.h.h.HttpServer2 HttpServer Acceptor: isRunning is false. Rechecking.
[junit4] 2> 9212609 WARN (1660392323@qtp-1808189085-1 - Acceptor0 HttpServer2$SelectChannelConnectorWithSafeStartup@lucene2-us-west.apache.org:33736) [ ] o.a.h.h.HttpServer2 HttpServer Acceptor: isRunning is false
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsUnloadDistributedZkTest_85F6F73912499F48-001
[junit4] 2> Aug 25, 2017 6:53:12 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 34 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {other_tl1=FST50, range_facet_l_dv=PostingsFormat(name=LuceneVarGapFixedInterval), multiDefault=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), a_t=FST50, intDefault=FST50, id_i1=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), a_i1=FST50, id=PostingsFormat(name=LuceneVarGapFixedInterval), range_facet_i_dv=FST50, text=PostingsFormat(name=Memory), range_facet_l=FST50, timestamp=FST50}, docValues:{other_tl1=DocValuesFormat(name=Direct), range_facet_l_dv=DocValuesFormat(name=Asserting), multiDefault=DocValuesFormat(name=Memory), intDefault=DocValuesFormat(name=Direct), a_i1=DocValuesFormat(name=Direct), range_facet_l=DocValuesFormat(name=Direct), _version_=DocValuesFormat(name=Direct), a_t=DocValuesFormat(name=Direct), id_i1=DocValuesFormat(name=Memory), range_facet_i_dv=DocValuesFormat(name=Direct), id=DocValuesFormat(name=Asserting), text=DocValuesFormat(name=Lucene70), intDvoDefault=DocValuesFormat(name=Lucene70), timestamp=DocValuesFormat(name=Direct)}, maxPointsInLeafNode=1922, maxMBSortInHeap=7.843133657731933, sim=RandomSimilarity(queryNorm=false): {}, locale=ar-TN, timezone=Europe/Belgrade
[junit4] 2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=2,free=200056608,total=525860864
[junit4] 2> NOTE: All tests run in this JVM: [MissingSegmentRecoveryTest, SolrIndexConfigTest, V2StandaloneTest, OutputWriterTest, TestUnifiedSolrHighlighter, TestOrdValues, SuggestComponentContextFilterQueryTest, TestAnalyzeInfixSuggestions, SolrCloudExampleTest, AtomicUpdatesTest, ConfigureRecoveryStrategyTest, TestSolrConfigHandler, TestHashQParserPlugin, DocValuesNotIndexedTest, CustomCollectionTest, TestPHPSerializedResponseWriter, OutOfBoxZkACLAndCredentialsProvidersTest, CdcrReplicationDistributedZkTest, TestSolrCloudWithDelegationTokens, TestDocumentBuilder, TestSolrFieldCacheBean, LargeFieldTest, TestHdfsCloudBackupRestore, TestSegmentSorting, TestBinaryField, TestManagedStopFilterFactory, TestFastOutputStream, TestConfigSetsAPIZkFailure, RemoteQueryErrorTest, TestConfigsApi, TestSQLHandler, JSONWriterTest, SpellCheckCollatorTest, TestIBSimilarityFactory, MoveReplicaTest, TestFieldResource, TestRealTimeGet, TestNumericTerms32, TestRandomFaceting, TestExpandComponent, HdfsBasicDistributedZk2Test, OpenExchangeRatesOrgProviderTest, CursorPagingTest, TestSolrCloudWithHadoopAuthPlugin, IgnoreCommitOptimizeUpdateProcessorFactoryTest, DeleteStatusTest, SpellingQueryConverterTest, TestJettySolrRunner, TestStreamBody, TestNumericTokenStream, SolrIndexSplitterTest, TestTrackingShardHandlerFactory, CollectionsAPIAsyncDistributedZkTest, TestRandomDVFaceting, TestFieldCacheVsDocValues, TestRangeQuery, TestSchemaVersionResource, TestRemoteStreaming, MetricUtilsTest, TestWriterPerf, TestManagedSchemaThreadSafety, TestReplicationHandler, TestSolrCoreSnapshots, TestReloadDeadlock, SortSpecParsingTest, TestReplicationHandlerBackup, FieldMutatingUpdateProcessorTest, TestStressCloudBlindAtomicUpdates, BasicAuthIntegrationTest, RAMDirectoryFactoryTest, TestSchemalessBufferedUpdates, BinaryUpdateRequestHandlerTest, SuggesterWFSTTest, TolerantUpdateProcessorTest, TestGraphMLResponseWriter, TestMaxScoreQueryParser, TestScoreJoinQPNoScore, HdfsLockFactoryTest, TestHighlightDedupGrouping, TestSubQueryTransformerDistrib, RequestHandlersTest, BlockDirectoryTest, DebugComponentTest, TestLegacyFieldReuse, WrapperMergePolicyFactoryTest, TestBadConfig, SyncSliceTest, TestInPlaceUpdatesDistrib, QueryElevationComponentTest, ConfigSetsAPITest, EnumFieldTest, OverseerTaskQueueTest, AliasIntegrationTest, XmlUpdateRequestHandlerTest, SolrCoreTest, HdfsChaosMonkeySafeLeaderTest, BitVectorTest, SolrMetricReporterTest, ResponseHeaderTest, TlogReplayBufferedWhileIndexingTest, TestFieldSortValues, TestSchemaNameResource, BlockJoinFacetDistribTest, TestFunctionQuery, TestJoin, CurrencyFieldTypeTest, FieldAnalysisRequestHandlerTest, TestMiniSolrCloudClusterSSL, DistribJoinFromCollectionTest, AnalyticsMergeStrategyTest, AddBlockUpdateTest, TestBlobHandler, ClassificationUpdateProcessorTest, TestAuthorizationFramework, ImplicitSnitchTest, TestCollectionsAPIViaSolrCloudCluster, AutoCommitTest, TestFaceting, TestHashPartitioner, DistributedSpellCheckComponentTest, TermVectorComponentDistributedTest, ZkControllerTest, TestStressReorder, TestReload, TestStressVersions, DistributedTermsComponentTest, TestCoreContainer, SimpleFacetsTest, TestSolr4Spatial, SpellCheckComponentTest, SolrCmdDistributorTest, PeerSyncTest, TestSort, BasicFunctionalityTest, TestLazyCores, DirectUpdateHandlerTest, DistributedQueryElevationComponentTest, SimplePostToolTest, TestCoreDiscovery, TestExtendedDismaxParser, SuggesterFSTTest, CoreAdminHandlerTest, SolrRequestParserTest, SuggesterTest, TestStressLucene, TestTrie, SpatialFilterTest, TestCSVLoader, PolyFieldTest, WordBreakSolrSpellCheckerTest, TestUpdate, TestAtomicUpdateErrorCases, TestWordDelimiterFilterFactory, DefaultValueUpdateProcessorTest, DistanceFunctionTest, TestSolrDeletionPolicy1, SolrInfoBeanTest, XsltUpdateRequestHandlerTest, CacheHeaderTest, IndexBasedSpellCheckerTest, TestSurroundQueryParser, LukeRequestHandlerTest, DisMaxRequestHandlerTest, TestQueryUtils, DirectSolrSpellCheckerTest, TestReversedWildcardFilterFactory, DocumentAnalysisRequestHandlerTest, TestQueryTypes, TestOmitPositions, DocumentBuilderTest, TestValueSourceCache, TestIndexingPerformance, MoreLikeThisHandlerTest, RequiredFieldsTest, TestSolrQueryParser, SolrPluginUtilsTest, TestCollationField, TestCSVResponseWriter, MBeansHandlerTest, QueryParsingTest, PingRequestHandlerTest, TestComponentsName, TestLFUCache, PreAnalyzedFieldTest, TestSystemIdResolver, TestSuggestSpellingConverter, ClusterStateTest, TestSolrJ, TestLRUCache, TestUtils, ZkNodePropsTest, SliceStateTest, SystemInfoHandlerTest, UUIDFieldTest, FileUtilsTest, DistributedMLTComponentTest, DistributedIntervalFacetingTest, ActionThrottleTest, AssignBackwardCompatibilityTest, CdcrVersionReplicationTest, ChaosMonkeyNothingIsSafeWithPullReplicasTest, CleanupOldIndexTest, CollectionReloadTest, CollectionStateFormat2Test, CollectionTooManyReplicasTest, ConnectionManagerTest, DeleteNodeTest, DistributedQueueTest, DistributedVersionInfoTest, MigrateRouteKeyTest, MoveReplicaHDFSTest, NodeMutatorTest, OverriddenZkACLAndCredentialsProvidersTest, OverseerModifyCollectionTest, RecoveryAfterSoftCommitTest, RestartWhileUpdatingTest, SharedFSAutoReplicaFailoverUtilsTest, SimpleCollectionCreateDeleteTest, SolrCLIZkUtilsTest, TestAuthenticationFramework, TestCloudDeleteByQuery, TestCloudPivotFacet, TestCloudPseudoReturnFields, TestCloudRecovery, TestCloudSearcherWarming, TestClusterProperties, TestConfigSetsAPI, TestExclusionRuleCollectionAccess, TestLeaderElectionZkExpiry, TestLocalFSCloudBackupRestore, TestPullReplicaErrorHandling, TestRandomRequestDistribution, TestRebalanceLeaders, TestTlogReplica, HdfsNNFailoverTest, HdfsRecoverLeaseTest, HdfsRecoveryZkTest, HdfsRestartWhileUpdatingTest, HdfsUnloadDistributedZkTest]
[junit4] Completed [567/731 (4!)] on J0 in 192.53s, 1 test, 1 error <<< FAILURES!
[...truncated 43879 lines...]
[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 33 - Still
unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/33/
8 tests failed.
FAILED: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.test
Error Message:
The Monkey ran for over 45 seconds and no jetties were stopped - this is worth investigating!
Stack Trace:
java.lang.AssertionError: The Monkey ran for over 45 seconds and no jetties were stopped - this is worth investigating!
at __randomizedtesting.SeedInfo.seed([AE887A61CD5492CE:26DC45BB63A8FF36]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.ChaosMonkey.stopTheMonkey(ChaosMonkey.java:587)
at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.test(ChaosMonkeySafeLeaderTest.java:133)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
Error Message:
2 threads leaked from SUITE scope at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest: 1) Thread[id=4167, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderTest] at java.lang.Thread.sleep(Native Method) at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135) 2) Thread[id=4168, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderTest] at java.lang.Thread.sleep(Native Method) at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest:
1) Thread[id=4167, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135)
2) Thread[id=4168, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135)
at __randomizedtesting.SeedInfo.seed([AE887A61CD5492CE]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=4167, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderTest] at org.apache.solr.common.util.ByteUtils.UTF16toUTF8(ByteUtils.java:90) at org.apache.solr.common.util.JavaBinCodec.writeStr(JavaBinCodec.java:840) at org.apache.solr.common.util.JavaBinCodec.writePrimitive(JavaBinCodec.java:932) at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:328) at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228) at org.apache.solr.common.util.JavaBinCodec.writeSolrInputDocument(JavaBinCodec.java:616) at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:355) at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228) at org.apache.solr.common.util.JavaBinCodec.writeMapEntry(JavaBinCodec.java:764) at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:383) at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228) at org.apache.solr.common.util.JavaBinCodec.writeIterator(JavaBinCodec.java:705) at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:367) at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228) at org.apache.solr.common.util.JavaBinCodec.writeNamedList(JavaBinCodec.java:223) at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:330) at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228) at org.apache.solr.common.util.JavaBinCodec.marshal(JavaBinCodec.java:155) at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.marshal(JavaBinUpdateRequestCodec.java:91) at org.apache.solr.client.solrj.impl.BinaryRequestWriter.getContentStream(BinaryRequestWriter.java:67) at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:94) at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:104) at org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:382) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195) at org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:176) at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:115) 2) Thread[id=4168, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderTest] at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:328) at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228) at org.apache.solr.common.util.JavaBinCodec.writeMapEntry(JavaBinCodec.java:764) at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:383) at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228) at org.apache.solr.common.util.JavaBinCodec.writeIterator(JavaBinCodec.java:705) at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:367) at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228) at org.apache.solr.common.util.JavaBinCodec.writeNamedList(JavaBinCodec.java:223) at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:330) at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228) at org.apache.solr.common.util.JavaBinCodec.marshal(JavaBinCodec.java:155) at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.marshal(JavaBinUpdateRequestCodec.java:91) at org.apache.solr.client.solrj.impl.BinaryRequestWriter.getContentStream(BinaryRequestWriter.java:67) at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:94) at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:104) at org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:382) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195) at org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:176) at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:115)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=4167, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderTest]
at org.apache.solr.common.util.ByteUtils.UTF16toUTF8(ByteUtils.java:90)
at org.apache.solr.common.util.JavaBinCodec.writeStr(JavaBinCodec.java:840)
at org.apache.solr.common.util.JavaBinCodec.writePrimitive(JavaBinCodec.java:932)
at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:328)
at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
at org.apache.solr.common.util.JavaBinCodec.writeSolrInputDocument(JavaBinCodec.java:616)
at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:355)
at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
at org.apache.solr.common.util.JavaBinCodec.writeMapEntry(JavaBinCodec.java:764)
at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:383)
at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
at org.apache.solr.common.util.JavaBinCodec.writeIterator(JavaBinCodec.java:705)
at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:367)
at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
at org.apache.solr.common.util.JavaBinCodec.writeNamedList(JavaBinCodec.java:223)
at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:330)
at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
at org.apache.solr.common.util.JavaBinCodec.marshal(JavaBinCodec.java:155)
at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.marshal(JavaBinUpdateRequestCodec.java:91)
at org.apache.solr.client.solrj.impl.BinaryRequestWriter.getContentStream(BinaryRequestWriter.java:67)
at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:94)
at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:104)
at org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:382)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195)
at org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:176)
at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:115)
2) Thread[id=4168, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderTest]
at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:328)
at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
at org.apache.solr.common.util.JavaBinCodec.writeMapEntry(JavaBinCodec.java:764)
at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:383)
at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
at org.apache.solr.common.util.JavaBinCodec.writeIterator(JavaBinCodec.java:705)
at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:367)
at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
at org.apache.solr.common.util.JavaBinCodec.writeNamedList(JavaBinCodec.java:223)
at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:330)
at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
at org.apache.solr.common.util.JavaBinCodec.marshal(JavaBinCodec.java:155)
at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.marshal(JavaBinUpdateRequestCodec.java:91)
at org.apache.solr.client.solrj.impl.BinaryRequestWriter.getContentStream(BinaryRequestWriter.java:67)
at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:94)
at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:104)
at org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:382)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195)
at org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:176)
at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:115)
at __randomizedtesting.SeedInfo.seed([AE887A61CD5492CE]:0)
FAILED: org.apache.solr.cloud.hdfs.HdfsBasicDistributedZkTest.test
Error Message:
Error from server at http://127.0.0.1:56609: ADDREPLICA failed to create replica
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:56609: ADDREPLICA failed to create replica
at __randomizedtesting.SeedInfo.seed([AE887A61CD5492CE:26DC45BB63A8FF36]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:627)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1121)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:862)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:793)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195)
at org.apache.solr.cloud.BasicDistributedZkTest.testANewCollectionInOneInstanceWithManualShardAssignement(BasicDistributedZkTest.java:801)
at org.apache.solr.cloud.BasicDistributedZkTest.test(BasicDistributedZkTest.java:374)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsBasicDistributedZkTest
Error Message:
3 threads leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsBasicDistributedZkTest: 1) Thread[id=16924, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HdfsBasicDistributedZkTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=16912, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HdfsBasicDistributedZkTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=16904, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HdfsBasicDistributedZkTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 3 threads leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsBasicDistributedZkTest:
1) Thread[id=16924, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HdfsBasicDistributedZkTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=16912, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HdfsBasicDistributedZkTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=16904, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HdfsBasicDistributedZkTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([AE887A61CD5492CE]:0)
FAILED: org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest.test
Error Message:
The Monkey ran for over 45 seconds and no jetties were stopped - this is worth investigating!
Stack Trace:
java.lang.AssertionError: The Monkey ran for over 45 seconds and no jetties were stopped - this is worth investigating!
at __randomizedtesting.SeedInfo.seed([AE887A61CD5492CE:26DC45BB63A8FF36]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.ChaosMonkey.stopTheMonkey(ChaosMonkey.java:587)
at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.test(ChaosMonkeySafeLeaderTest.java:133)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest
Error Message:
2 threads leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest: 1) Thread[id=15881, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-HdfsChaosMonkeySafeLeaderTest] at java.lang.Thread.sleep(Native Method) at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135) 2) Thread[id=15882, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-HdfsChaosMonkeySafeLeaderTest] at java.lang.Thread.sleep(Native Method) at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest:
1) Thread[id=15881, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-HdfsChaosMonkeySafeLeaderTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135)
2) Thread[id=15882, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-HdfsChaosMonkeySafeLeaderTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135)
at __randomizedtesting.SeedInfo.seed([AE887A61CD5492CE]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=15881, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-HdfsChaosMonkeySafeLeaderTest] at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:328) at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228) at org.apache.solr.common.util.JavaBinCodec.writeSolrInputDocument(JavaBinCodec.java:616) at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:355) at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228) at org.apache.solr.common.util.JavaBinCodec.writeMapEntry(JavaBinCodec.java:764) at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:383) at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228) at org.apache.solr.common.util.JavaBinCodec.writeIterator(JavaBinCodec.java:705) at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:367) at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228) at org.apache.solr.common.util.JavaBinCodec.writeNamedList(JavaBinCodec.java:223) at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:330) at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228) at org.apache.solr.common.util.JavaBinCodec.marshal(JavaBinCodec.java:155) at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.marshal(JavaBinUpdateRequestCodec.java:91) at org.apache.solr.client.solrj.impl.BinaryRequestWriter.getContentStream(BinaryRequestWriter.java:67) at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:94) at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:104) at org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:382) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195) at org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:176) at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:115) 2) Thread[id=15882, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-HdfsChaosMonkeySafeLeaderTest] at java.lang.Thread.sleep(Native Method) at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=15881, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-HdfsChaosMonkeySafeLeaderTest]
at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:328)
at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
at org.apache.solr.common.util.JavaBinCodec.writeSolrInputDocument(JavaBinCodec.java:616)
at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:355)
at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
at org.apache.solr.common.util.JavaBinCodec.writeMapEntry(JavaBinCodec.java:764)
at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:383)
at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
at org.apache.solr.common.util.JavaBinCodec.writeIterator(JavaBinCodec.java:705)
at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:367)
at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
at org.apache.solr.common.util.JavaBinCodec.writeNamedList(JavaBinCodec.java:223)
at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:330)
at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
at org.apache.solr.common.util.JavaBinCodec.marshal(JavaBinCodec.java:155)
at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.marshal(JavaBinUpdateRequestCodec.java:91)
at org.apache.solr.client.solrj.impl.BinaryRequestWriter.getContentStream(BinaryRequestWriter.java:67)
at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:94)
at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:104)
at org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:382)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195)
at org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:176)
at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:115)
2) Thread[id=15882, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-HdfsChaosMonkeySafeLeaderTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135)
at __randomizedtesting.SeedInfo.seed([AE887A61CD5492CE]:0)
Build Log:
[...truncated 11721 lines...]
[junit4] Suite: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
[junit4] 2> 878451 INFO (SUITE-ChaosMonkeySafeLeaderTest-seed#[AE887A61CD5492CE]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_AE887A61CD5492CE-001/init-core-data-001
[junit4] 2> 878451 WARN (SUITE-ChaosMonkeySafeLeaderTest-seed#[AE887A61CD5492CE]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
[junit4] 2> 878451 INFO (SUITE-ChaosMonkeySafeLeaderTest-seed#[AE887A61CD5492CE]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 878453 INFO (SUITE-ChaosMonkeySafeLeaderTest-seed#[AE887A61CD5492CE]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 878453 INFO (SUITE-ChaosMonkeySafeLeaderTest-seed#[AE887A61CD5492CE]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /_pk/g
[junit4] 2> 878470 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 878470 INFO (Thread-1009) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 878470 INFO (Thread-1009) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 878471 ERROR (Thread-1009) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 878570 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.c.ZkTestServer start zk server on port:38853
[junit4] 2> 878582 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 878584 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 878585 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 878586 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 878587 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 878587 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 878588 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 878589 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 878590 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 878591 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 878592 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 878593 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
[junit4] 2> 878698 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 878699 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2d4151dd{/_pk/g,null,AVAILABLE}
[junit4] 2> 878700 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1b91e9f1{HTTP/1.1,[http/1.1]}{127.0.0.1:56567}
[junit4] 2> 878700 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.e.j.s.Server Started @884771ms
[junit4] 2> 878700 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_AE887A61CD5492CE-001/tempDir-001/control/data, replicaType=NRT, hostContext=/_pk/g, hostPort=56567, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_AE887A61CD5492CE-001/control-001/cores}
[junit4] 2> 878700 ERROR (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 878700 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.1.0
[junit4] 2> 878700 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 878700 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 878700 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-22T16:37:26.333Z
[junit4] 2> 878703 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 878703 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_AE887A61CD5492CE-001/control-001/solr.xml
[junit4] 2> 878708 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@344b084c, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 878710 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38853/solr
[junit4] 2> 878762 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [n:127.0.0.1:56567__pk%2Fg ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 878762 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [n:127.0.0.1:56567__pk%2Fg ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:56567__pk%2Fg
[junit4] 2> 878762 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [n:127.0.0.1:56567__pk%2Fg ] o.a.s.c.Overseer Overseer (id=98528123034271748-127.0.0.1:56567__pk%2Fg-n_0000000000) starting
[junit4] 2> 878767 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [n:127.0.0.1:56567__pk%2Fg ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56567__pk%2Fg
[junit4] 2> 878782 INFO (OverseerStateUpdate-98528123034271748-127.0.0.1:56567__pk%2Fg-n_0000000000) [n:127.0.0.1:56567__pk%2Fg ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 878919 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [n:127.0.0.1:56567__pk%2Fg ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@344b084c
[junit4] 2> 878927 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [n:127.0.0.1:56567__pk%2Fg ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@344b084c
[junit4] 2> 878927 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [n:127.0.0.1:56567__pk%2Fg ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@344b084c
[junit4] 2> 878928 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [n:127.0.0.1:56567__pk%2Fg ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_AE887A61CD5492CE-001/control-001/cores
[junit4] 2> 878948 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 878948 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:38853/solr ready
[junit4] 2> 878949 INFO (qtp1168414568-3730) [n:127.0.0.1:56567__pk%2Fg ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:56567__pk%252Fg&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 878955 INFO (OverseerThreadFactory-1396-thread-1-processing-n:127.0.0.1:56567__pk%2Fg) [n:127.0.0.1:56567__pk%2Fg ] o.a.s.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 879062 INFO (qtp1168414568-3728) [n:127.0.0.1:56567__pk%2Fg ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 879062 INFO (qtp1168414568-3728) [n:127.0.0.1:56567__pk%2Fg ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 879168 INFO (zkCallback-564-thread-1-processing-n:127.0.0.1:56567__pk%2Fg) [n:127.0.0.1:56567__pk%2Fg ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 880082 INFO (qtp1168414568-3728) [n:127.0.0.1:56567__pk%2Fg c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 880093 INFO (qtp1168414568-3728) [n:127.0.0.1:56567__pk%2Fg c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
[junit4] 2> 880175 INFO (qtp1168414568-3728) [n:127.0.0.1:56567__pk%2Fg c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 880188 INFO (qtp1168414568-3728) [n:127.0.0.1:56567__pk%2Fg c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
[junit4] 2> 880189 INFO (qtp1168414568-3728) [n:127.0.0.1:56567__pk%2Fg c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@344b084c
[junit4] 2> 880196 INFO (qtp1168414568-3728) [n:127.0.0.1:56567__pk%2Fg c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 880196 INFO (qtp1168414568-3728) [n:127.0.0.1:56567__pk%2Fg c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_AE887A61CD5492CE-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_AE887A61CD5492CE-001/control-001/cores/control_collection_shard1_replica_n1/data/]
[junit4] 2> 880199 INFO (qtp1168414568-3728) [n:127.0.0.1:56567__pk%2Fg c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=26, maxMergeAtOnceExplicit=39, maxMergedSegmentMB=12.4951171875, floorSegmentMB=0.3984375, forceMergeDeletesPctAllowed=4.877581534016178, segmentsPerTier=11.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7470195475198435
[junit4] 2> 880219 WARN (qtp1168414568-3728) [n:127.0.0.1:56567__pk%2Fg c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 880258 INFO (qtp1168414568-3728) [n:127.0.0.1:56567__pk%2Fg c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 880258 INFO (qtp1168414568-3728) [n:127.0.0.1:56567__pk%2Fg c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 880259 INFO (qtp1168414568-3728) [n:127.0.0.1:56567__pk%2Fg c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 15000ms;
[junit4] 2> 880259 INFO (qtp1168414568-3728) [n:127.0.0.1:56567__pk%2Fg c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 880260 INFO (qtp1168414568-3728) [n:127.0.0.1:56567__pk%2Fg c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@52c6bad6
[junit4] 2> 880261 INFO (qtp1168414568-3728) [n:127.0.0.1:56567__pk%2Fg c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@447f0ebb[control_collection_shard1_replica_n1] main]
[junit4] 2> 880262 INFO (qtp1168414568-3728) [n:127.0.0.1:56567__pk%2Fg c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 880262 INFO (qtp1168414568-3728) [n:127.0.0.1:56567__pk%2Fg c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 880263 INFO (qtp1168414568-3728) [n:127.0.0.1:56567__pk%2Fg c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 880263 INFO (qtp1168414568-3728) [n:127.0.0.1:56567__pk%2Fg c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1576449970427396096
[junit4] 2> 880264 INFO (searcherExecutor-1399-thread-1-processing-n:127.0.0.1:56567__pk%2Fg x:control_collection_shard1_replica_n1 s:shard1 c:control_collection) [n:127.0.0.1:56567__pk%2Fg c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@447f0ebb[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 880269 INFO (qtp1168414568-3728) [n:127.0.0.1:56567__pk%2Fg c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 880269 INFO (qtp1168414568-3728) [n:127.0.0.1:56567__pk%2Fg c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 880269 INFO (qtp1168414568-3728) [n:127.0.0.1:56567__pk%2Fg c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:56567/_pk/g/control_collection_shard1_replica_n1/
[junit4] 2> 880269 INFO (qtp1168414568-3728) [n:127.0.0.1:56567__pk%2Fg c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 880270 INFO (qtp1168414568-3728) [n:127.0.0.1:56567__pk%2Fg c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:56567/_pk/g/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 880271 INFO (qtp1168414568-3728) [n:127.0.0.1:56567__pk%2Fg c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 880273 INFO (qtp1168414568-3728) [n:127.0.0.1:56567__pk%2Fg c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:56567/_pk/g/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 880376 INFO (zkCallback-564-thread-1-processing-n:127.0.0.1:56567__pk%2Fg) [n:127.0.0.1:56567__pk%2Fg ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 880378 INFO (qtp1168414568-3728) [n:127.0.0.1:56567__pk%2Fg c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 880381 INFO (qtp1168414568-3728) [n:127.0.0.1:56567__pk%2Fg c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1319
[junit4] 2> 880386 INFO (qtp1168414568-3730) [n:127.0.0.1:56567__pk%2Fg ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 880482 INFO (zkCallback-564-thread-1-processing-n:127.0.0.1:56567__pk%2Fg) [n:127.0.0.1:56567__pk%2Fg ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 880958 INFO (OverseerCollectionConfigSetProcessor-98528123034271748-127.0.0.1:56567__pk%2Fg-n_0000000000) [n:127.0.0.1:56567__pk%2Fg ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 881386 INFO (qtp1168414568-3730) [n:127.0.0.1:56567__pk%2Fg ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:56567__pk%252Fg&wt=javabin&version=2} status=0 QTime=2436
[junit4] 2> 881394 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 881395 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:38853/solr ready
[junit4] 2> 881395 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 881396 INFO (qtp1168414568-3727) [n:127.0.0.1:56567__pk%2Fg ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=5&createNodeSet=&stateFormat=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 881397 INFO (OverseerThreadFactory-1396-thread-2-processing-n:127.0.0.1:56567__pk%2Fg) [n:127.0.0.1:56567__pk%2Fg ] o.a.s.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 881398 WARN (OverseerThreadFactory-1396-thread-2-processing-n:127.0.0.1:56567__pk%2Fg) [n:127.0.0.1:56567__pk%2Fg ] o.a.s.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
[junit4] 2> 881511 INFO (qtp1168414568-3727) [n:127.0.0.1:56567__pk%2Fg ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 881511 INFO (qtp1168414568-3727) [n:127.0.0.1:56567__pk%2Fg ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=5&createNodeSet=&stateFormat=2&wt=javabin&version=2} status=0 QTime=115
[junit4] 2> 881687 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_AE887A61CD5492CE-001/shard-1-001 of type NRT
[junit4] 2> 881688 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 881689 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6eced090{/_pk/g,null,AVAILABLE}
[junit4] 2> 881689 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1a24b7f7{HTTP/1.1,[http/1.1]}{127.0.0.1:44829}
[junit4] 2> 881689 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.e.j.s.Server Started @887760ms
[junit4] 2> 881689 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_AE887A61CD5492CE-001/tempDir-001/jetty1, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/_pk/g, hostPort=44829, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_AE887A61CD5492CE-001/shard-1-001/cores}
[junit4] 2> 881689 ERROR (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 881689 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.1.0
[junit4] 2> 881690 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 881690 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 881690 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-22T16:37:29.323Z
[junit4] 2> 881736 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 881736 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_AE887A61CD5492CE-001/shard-1-001/solr.xml
[junit4] 2> 881740 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@344b084c, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 881745 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38853/solr
[junit4] 2> 881754 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [n:127.0.0.1:44829__pk%2Fg ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 881755 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [n:127.0.0.1:44829__pk%2Fg ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 881757 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [n:127.0.0.1:44829__pk%2Fg ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44829__pk%2Fg
[junit4] 2> 881757 INFO (zkCallback-571-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 881758 INFO (zkCallback-564-thread-1-processing-n:127.0.0.1:56567__pk%2Fg) [n:127.0.0.1:56567__pk%2Fg ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 881771 INFO (zkCallback-576-thread-2-processing-n:127.0.0.1:44829__pk%2Fg) [n:127.0.0.1:44829__pk%2Fg ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 881804 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [n:127.0.0.1:44829__pk%2Fg ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@344b084c
[junit4] 2> 881812 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [n:127.0.0.1:44829__pk%2Fg ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@344b084c
[junit4] 2> 881812 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [n:127.0.0.1:44829__pk%2Fg ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@344b084c
[junit4] 2> 881814 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [n:127.0.0.1:44829__pk%2Fg ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_AE887A61CD5492CE-001/shard-1-001/cores
[junit4] 2> 881875 INFO (qtp257378453-3771) [n:127.0.0.1:44829__pk%2Fg ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:44829__pk%252Fg&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 881876 INFO (OverseerCollectionConfigSetProcessor-98528123034271748-127.0.0.1:56567__pk%2Fg-n_0000000000) [n:127.0.0.1:56567__pk%2Fg ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 881877 INFO (OverseerThreadFactory-1396-thread-3-processing-n:127.0.0.1:56567__pk%2Fg) [n:127.0.0.1:56567__pk%2Fg ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:44829__pk%2Fg for creating new replica
[junit4] 2> 881879 INFO (qtp257378453-3769) [n:127.0.0.1:44829__pk%2Fg ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n101&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 881880 INFO (qtp257378453-3769) [n:127.0.0.1:44829__pk%2Fg ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 881983 INFO (zkCallback-576-thread-2-processing-n:127.0.0.1:44829__pk%2Fg) [n:127.0.0.1:44829__pk%2Fg ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 882892 INFO (qtp257378453-3769) [n:127.0.0.1:44829__pk%2Fg c:collection1 s:shard2 x:collection1_shard2_replica_n101] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 882902 INFO (qtp257378453-3769) [n:127.0.0.1:44829__pk%2Fg c:collection1 s:shard2 x:collection1_shard2_replica_n101] o.a.s.s.IndexSchema [collection1_shard2_replica_n101] Schema name=test
[junit4] 2> 882981 INFO (qtp257378453-3769) [n:127.0.0.1:44829__pk%2Fg c:collection1 s:shard2 x:collection1_shard2_replica_n101] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 883000 INFO (qtp257378453-3769) [n:127.0.0.1:44829__pk%2Fg c:collection1 s:shard2 x:collection1_shard2_replica_n101] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n101' using configuration from collection collection1, trusted=true
[junit4] 2> 883000 INFO (qtp257378453-3769) [n:127.0.0.1:44829__pk%2Fg c:collection1 s:shard2 x:collection1_shard2_replica_n101] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_n101' (registry 'solr.core.collection1.shard2.replica_n101') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@344b084c
[junit4] 2> 883000 INFO (qtp257378453-3769) [n:127.0.0.1:44829__pk%2Fg c:collection1 s:shard2 x:collection1_shard2_replica_n101] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 883001 INFO (qtp257378453-3769) [n:127.0.0.1:44829__pk%2Fg c:collection1 s:shard2 x:collection1_shard2_replica_n101] o.a.s.c.SolrCore [[collection1_shard2_replica_n101] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_AE887A61CD5492CE-001/shard-1-001/cores/collection1_shard2_replica_n101], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_AE887A61CD5492CE-001/shard-1-001/cores/collection1_shard2_replica_n101/data/]
[junit4] 2> 883003 INFO (qtp257378453-3769) [n:127.0.0.1:44829__pk%2Fg c:collection1 s:shard2 x:collection1_shard2_replica_n101] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=26, maxMergeAtOnceExplicit=39, maxMergedSegmentMB=12.4951171875, floorSegmentMB=0.3984375, forceMergeDeletesPctAllowed=4.877581534016178, segmentsPerTier=11.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7470195475198435
[junit4] 2> 883011 WARN (qtp257378453-3769) [n:127.0.0.1:44829__pk%2Fg c:collection1 s:shard2 x:collection1_shard2_replica_n101] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 883043 INFO (qtp257378453-3769) [n:127.0.0.1:44829__pk%2Fg c:collection1 s:shard2 x:collection1_shard2_replica_n101] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 883043 INFO (qtp257378453-3769) [n:127.0.0.1:44829__pk%2Fg c:collection1 s:shard2 x:collection1_shard2_replica_n101] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 883044 INFO (qtp257378453-3769) [n:127.0.0.1:44829__pk%2Fg c:collection1 s:shard2 x:collection1_shard2_replica_n101] o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 15000ms;
[junit4] 2> 883044 INFO (qtp257378453-3769) [n:127.0.0.1:44829__pk%2Fg c:collection1 s:shard2 x:collection1_shard2_replica_n101] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 883045 INFO (qtp257378453-3769) [n:127.0.0.1:44829__pk%2Fg c:collection1 s:shard2 x:collection1_shard2_replica_n101] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@4d4f6e65
[junit4] 2> 883045 INFO (qtp257378453-3769) [n:127.0.0.1:44829__pk%2Fg c:collection1 s:shard2 x:collection1_shard2_replica_n101] o.a.s.s.SolrIndexSearcher Opening [Searcher@1cb3553d[collection1_shard2_replica_n101] main]
[junit4] 2> 883046 INFO (qtp257378453-3769) [n:127.0.0.1:44829__pk%2Fg c:collection1 s:shard2 x:collection1_shard2_replica_n101] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 883047 INFO (qtp257378453-3769) [n:127.0.0.1:44829__pk%2Fg c:collection1 s:shard2 x:collection1_shard2_replica_n101] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 883047 INFO (qtp257378453-3769) [n:127.0.0.1:44829__pk%2Fg c:collection1 s:shard2 x:collection1_shard2_replica_n101] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 883047 INFO (qtp257378453-3769) [n:127.0.0.1:44829__pk%2Fg c:collection1 s:shard2 x:collection1_shard2_replica_n101] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1576449973346631680
[junit4] 2> 883050 INFO (searcherExecutor-1410-thread-1-processing-n:127.0.0.1:44829__pk%2Fg x:collection1_shard2_replica_n101 s:shard2 c:collection1) [n:127.0.0.1:44829__pk%2Fg c:collection1 s:shard2 x:collection1_shard2_replica_n101] o.a.s.c.SolrCore [collection1_shard2_replica_n101] Registered new searcher Searcher@1cb3553d[collection1_shard2_replica_n101] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 883053 INFO (qtp257378453-3769) [n:127.0.0.1:44829__pk%2Fg c:collection1 s:shard2 x:collection1_shard2_replica_n101] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 883053 INFO (qtp257378453-3769) [n:127.0.0.1:44829__pk%2Fg c:collection1 s:shard2 x:collection1_shard2_replica_n101] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 883053 INFO (qtp257378453-3769) [n:127.0.0.1:44829__pk%2Fg c:collection1 s:shard2 x:collection1_shard2_replica_n101] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:44829/_pk/g/collection1_shard2_replica_n101/
[junit4] 2> 883054 INFO (qtp257378453-3769) [n:127.0.0.1:44829__pk%2Fg c:collection1 s:shard2 x:collection1_shard2_replica_n101] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 883054 INFO (qtp257378453-3769) [n:127.0.0.1:44829__pk%2Fg c:collection1 s:shard2 x:collection1_shard2_replica_n101] o.a.s.c.SyncStrategy http://127.0.0.1:44829/_pk/g/collection1_shard2_replica_n101/ has no replicas
[junit4] 2> 883054 INFO (qtp257378453-3769) [n:127.0.0.1:44829__pk%2Fg c:collection1 s:shard2 x:collection1_shard2_replica_n101] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 883059 INFO (qtp257378453-3769) [n:127.0.0.1:44829__pk%2Fg c:collection1 s:shard2 x:collection1_shard2_replica_n101] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:44829/_pk/g/collection1_shard2_replica_n101/ shard2
[junit4] 2> 883163 INFO (zkCallback-576-thread-2-processing-n:127.0.0.1:44829__pk%2Fg) [n:127.0.0.1:44829__pk%2Fg ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 883211 INFO (qtp257378453-3769) [n:127.0.0.1:44829__pk%2Fg c:collection1 s:shard2 x:collection1_shard2_replica_n101] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 883212 INFO (qtp257378453-3769) [n:127.0.0.1:44829__pk%2Fg c:collection1 s:shard2 x:collection1_shard2_replica_n101] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n101&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1332
[junit4] 2> 883215 INFO (qtp257378453-3771) [n:127.0.0.1:44829__pk%2Fg ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:44829__pk%252Fg&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2} status=0 QTime=1340
[junit4] 2> 883317 INFO (zkCallback-576-thread-2-processing-n:127.0.0.1:44829__pk%2Fg) [n:127.0.0.1:44829__pk%2Fg ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 883346 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_AE887A61CD5492CE-001/shard-2-001 of type NRT
[junit4] 2> 883346 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 883348 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6fe276aa{/_pk/g,null,AVAILABLE}
[junit4] 2> 883348 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@46b60caf{HTTP/1.1,[http/1.1]}{127.0.0.1:53914}
[junit4] 2> 883348 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.e.j.s.Server Started @889420ms
[junit4] 2> 883349 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_AE887A61CD5492CE-001/tempDir-001/jetty2, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/_pk/g, hostPort=53914, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_AE887A61CD5492CE-001/shard-2-001/cores}
[junit4] 2> 883349 ERROR (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 883350 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.1.0
[junit4] 2> 883351 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 883351 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 883351 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-22T16:37:30.984Z
[junit4] 2> 883354 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 883354 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_AE887A61CD5492CE-001/shard-2-001/solr.xml
[junit4] 2> 883358 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@344b084c, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 883361 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38853/solr
[junit4] 2> 883374 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [n:127.0.0.1:53914__pk%2Fg ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 883375 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [n:127.0.0.1:53914__pk%2Fg ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 883375 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [n:127.0.0.1:53914__pk%2Fg ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53914__pk%2Fg
[junit4] 2> 883376 INFO (zkCallback-571-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 883377 INFO (zkCallback-564-thread-2-processing-n:127.0.0.1:56567__pk%2Fg) [n:127.0.0.1:56567__pk%2Fg ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 883376 INFO (zkCallback-576-thread-2-processing-n:127.0.0.1:44829__pk%2Fg) [n:127.0.0.1:44829__pk%2Fg ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 883398 INFO (zkCallback-582-thread-2-processing-n:127.0.0.1:53914__pk%2Fg) [n:127.0.0.1:53914__pk%2Fg ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 883494 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [n:127.0.0.1:53914__pk%2Fg ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@344b084c
[junit4] 2> 883503 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [n:127.0.0.1:53914__pk%2Fg ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@344b084c
[junit4] 2> 883503 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [n:127.0.0.1:53914__pk%2Fg ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@344b084c
[junit4] 2> 883504 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[AE887A61CD5492CE]) [n:127.0.0.1:53914__pk%2Fg ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_AE887A61CD5492CE-001/shard-2-001/cores
[junit4] 2> 883537 INFO (qtp257378453-3772) [n:127.0.0.1:44829__pk%2Fg ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:53914__pk%252Fg&action=ADDREPLICA&collection=collection1&shard=shard3&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 883558 INFO (OverseerCollectionConfigSetProcessor-98528123034271748-127.0.0.1:56567__pk%2Fg-n_0000000000) [n:127.0.0.1:56567__pk%2Fg ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 883570 INFO (OverseerThreadFactory-1396-thread-4-processing-n:127.0.0.1:56567__pk%2Fg) [n:127.0.0.1:56567__pk%2Fg ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:53914__pk%2Fg for creating new replica
[junit4] 2> 883576 INFO (qtp1218089458-3802) [n:127.0.0.1:53914__pk%2Fg ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard3_replica_n103&action=CREATE&collection=collection1&shard=shard3&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 883576 INFO (qtp1218089458-3802) [n:127.0.0.1:53914__pk%2Fg ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 883679 INFO (zkCallback-576-thread-2-processing-n:127.0.0.1:44829__pk%2Fg) [n:127.0.0.1:44829__pk%2Fg ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 883679 INFO (zkCallback-582-thread-2-processing-n:127.0.0.1:53914__pk%2Fg) [n:127.0.0.1:53914__pk%2Fg ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 884590 INFO (qtp1218089458-3802) [n:127.0.0.1:53914__pk%2Fg c:collection1 s:shard3 x:collection1_shard3_replica_n103] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 884609 INFO (qtp1218089458-3802) [n:127.0.0.1:53914__pk%2Fg c:collection1 s:shard3 x:collection1_shard3_replica_n103] o.a.s.s.IndexSchema [collection1_shard3_replica_n103] Schema name=test
[junit4] 2> 884692 INFO (qtp1218089458-3802) [n:127.0.0.1:53914__pk%2Fg c:collection1 s:shard3 x:collection1_shard3_replica_n103] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 884707 INFO (qtp1218089458-3802) [n:127.0.0.1:53914__pk%2Fg c:collection1 s:shard3 x:collection1_shard3_replica_n103] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard3_replica_n103' using configuration from collection collection1, trusted=true
[junit4] 2> 884708 INFO (qtp1218089458-3802) [n:127.0.0.1:53914__pk%2Fg c:collection1 s:shard3 x:collection1_shard3_replica_n103] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard3.replica_n103' (registry 'solr.core.collection1.shard3.replica_n103') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@344b084c
[junit4] 2> 884708 INFO (qtp1218089458-3802) [n:127.0.0.1:53914__pk%2Fg c:collection1 s:shard3 x:collection1_shard3_replica_n103] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 884708 INFO (qtp1218089458-3802) [n:127.0.0.1:53914__pk%2Fg c:collection1 s:shard3 x:collection1_shard3_replica_n103] o.a.s.c.SolrCore [[collection1_shard3_replica_n103] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_AE887A61CD5492CE-001/shard-2-001/cores/collection1_shard3_replica_n103], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_AE887A61CD5492CE-001/shard-2-001/cores/collection1_shard3_replica_n103/data/]
[junit4] 2> 884717 INFO (qtp1218089458-3802) [n:127.0.0.1:53914__pk%2Fg c:collection1 s:shard3 x:collection1_shard3_replica_n103] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=26, maxMergeAtOnceExplicit=39, maxMergedSegmentMB=12.4951171875, floorSegmentMB=0.3984375, forceMergeDeletesPctAllowed=4.877581534016178, segmentsPerTier=11.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7470195475198435
[junit4] 2> 884733 WARN (qtp1218089458-3802) [n:127.0.0.1:53914__pk%2Fg c:collection1 s:shard3 x:collection1_shard3_replica_n103] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 884769 INFO (qtp1218089458-3802) [n:127.0.0.1:53914__pk%2Fg c:collection1 s:shard3 x:collection1_shard3_replica_n103] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 884769 INFO (qtp1218089458-3802) [n:127.0.0.1:53914__pk%2Fg c:collection1 s:shard3 x:collection1_shard3_replica_n103] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 884770 INFO (qtp1218089458-3802) [n:127.0.0.1:53914__pk%2Fg c:collection1 s:shard3 x:collection1_shard3_replica_n103] o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 15000ms;
[junit4] 2> 884770 INFO (qtp1218089458-3802) [n:127.0.0.1:53914__pk%2Fg c:collection1 s:shard3 x:collection1_shard3_replica_n103] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 884771 INFO (qtp1218089458-3802) [n:127.0.0.1:53914__pk%2Fg c:collection1 s:shard3 x:collection1_shard3_replica_n103] o.a.s.u.RandomMergePolicy Ran
[...truncated too long message...]
492CE-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AE887A61CD5492CE-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to localhost/127.0.0.1:43801) [ ] o.a.h.h.s.d.IncrementalBlockReportManager IncrementalBlockReportManager interrupted
[junit4] 2> 2564373 WARN (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AE887A61CD5492CE-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AE887A61CD5492CE-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to localhost/127.0.0.1:43801) [ ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-81103478-10.41.0.5-1503421441026 (Datanode Uuid 2cf779a4-faaf-429b-a7c8-f722a9167b30) service to localhost/127.0.0.1:43801
[junit4] 2> 2564381 INFO (SUITE-HdfsBasicDistributedZkTest-seed#[AE887A61CD5492CE]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.hdfs.HdfsBasicDistributedZkTest_AE887A61CD5492CE-001
[junit4] 2> Aug 22, 2017 5:05:30 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 22 leaked thread(s).
[junit4] 2> Aug 22, 2017 5:05:50 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 3 threads leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsBasicDistributedZkTest:
[junit4] 2> 1) Thread[id=16924, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HdfsBasicDistributedZkTest]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 2) Thread[id=16912, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HdfsBasicDistributedZkTest]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 3) Thread[id=16904, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HdfsBasicDistributedZkTest]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Aug 22, 2017 5:05:50 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=16924, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HdfsBasicDistributedZkTest]
[junit4] 2> 2) Thread[id=16912, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HdfsBasicDistributedZkTest]
[junit4] 2> 3) Thread[id=16904, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HdfsBasicDistributedZkTest]
[junit4] 2> Aug 22, 2017 5:05:50 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: All leaked threads terminated.
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {other_tl1=BlockTreeOrds(blocksize=128), range_facet_l_dv=PostingsFormat(name=Memory), SubjectTerms_mfacet=BlockTreeOrds(blocksize=128), n_l1=Lucene50(blocksize=128), multiDefault=Lucene50(blocksize=128), intDefault=BlockTreeOrds(blocksize=128), n_td1=BlockTreeOrds(blocksize=128), n_d1=Lucene50(blocksize=128), range_facet_l=Lucene50(blocksize=128), n_f1=BlockTreeOrds(blocksize=128), n_tl1=BlockTreeOrds(blocksize=128), oddField_s=PostingsFormat(name=Memory), n_tf1=Lucene50(blocksize=128), a_t=Lucene50(blocksize=128), id=PostingsFormat(name=Memory), text=PostingsFormat(name=Direct), timestamp=Lucene50(blocksize=128), foo_i=BlockTreeOrds(blocksize=128), val_i=PostingsFormat(name=Memory), n_dt1=BlockTreeOrds(blocksize=128), a_i1=Lucene50(blocksize=128), n_ti1=PostingsFormat(name=Memory), rnd_b=PostingsFormat(name=Memory), foo_b=PostingsFormat(name=Direct), n_tdt1=BlockTreeOrds(blocksize=128), id_i1=BlockTreeOrds(blocksize=128), foo_d=PostingsFormat(name=Memory), range_facet_i_dv=Lucene50(blocksize=128), foo_f=PostingsFormat(name=Direct)}, docValues:{foo_i=DocValuesFormat(name=Lucene70), other_tl1=DocValuesFormat(name=Lucene70), range_facet_l_dv=DocValuesFormat(name=Direct), n_l1=DocValuesFormat(name=Lucene70), val_i=DocValuesFormat(name=Direct), intDefault=DocValuesFormat(name=Lucene70), n_dt1=DocValuesFormat(name=Lucene70), a_i1=DocValuesFormat(name=Lucene70), n_td1=DocValuesFormat(name=Lucene70), n_d1=DocValuesFormat(name=Lucene70), range_facet_l=DocValuesFormat(name=Lucene70), n_f1=DocValuesFormat(name=Lucene70), n_ti1=DocValuesFormat(name=Direct), n_tl1=DocValuesFormat(name=Lucene70), _version_=DocValuesFormat(name=Lucene70), n_tf1=DocValuesFormat(name=Lucene70), n_tdt1=DocValuesFormat(name=Lucene70), id_i1=DocValuesFormat(name=Lucene70), foo_d=DocValuesFormat(name=Direct), range_facet_i_dv=DocValuesFormat(name=Lucene70), intDvoDefault=DocValuesFormat(name=Direct), foo_f=DocValuesFormat(name=Memory), timestamp=DocValuesFormat(name=Lucene70)}, maxPointsInLeafNode=733, maxMBSortInHeap=5.211155181173087, sim=RandomSimilarity(queryNorm=false): {}, locale=da, timezone=Europe/Oslo
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=3,free=250257968,total=525860864
[junit4] 2> NOTE: All tests run in this JVM: [ClassificationUpdateProcessorFactoryTest, BasicDistributedZkTest, TestDocSet, HdfsDirectoryFactoryTest, LeaderInitiatedRecoveryOnShardRestartTest, JavabinLoaderTest, TestCoreAdminApis, TestFilteredDocIdSet, DistributedQueryElevationComponentTest, ShufflingReplicaListTransformerTest, SpellCheckCollatorTest, ConnectionReuseTest, TestPKIAuthenticationPlugin, TestCloudNestedDocsSort, TestTolerantUpdateProcessorCloud, SuggestComponentTest, CleanupOldIndexTest, SyncSliceTest, TestCollationField, TestIndexingPerformance, DistributedFacetExistsSmallTest, SmileWriterTest, HighlighterConfigTest, TestBinaryField, TestTestInjection, SimpleMLTQParserTest, TestScoreJoinQPScore, TestComponentsName, TestFieldCacheWithThreads, TestComplexPhraseQParserPlugin, DeleteReplicaTest, AutoScalingHandlerTest, UUIDFieldTest, TestWriterPerf, TestCrossCoreJoin, TestLocalFSCloudBackupRestore, DistributedQueueTest, TestDefaultStatsCache, TestDocumentBuilder, TestLMDirichletSimilarityFactory, TemplateUpdateProcessorTest, IndexBasedSpellCheckerTest, SolrIndexSplitterTest, PeerSyncWithIndexFingerprintCachingTest, ZkNodePropsTest, TestManagedResource, SolrPluginUtilsTest, CoreAdminCreateDiscoverTest, TestHighFrequencyDictionaryFactory, ChaosMonkeyNothingIsSafeWithPullReplicasTest, TestNumericTerms64, TestCryptoKeys, TestHighlightDedupGrouping, TestPhraseSuggestions, CloneFieldUpdateProcessorFactoryTest, TestSolrCloudWithKerberosAlt, TestNumericRangeQuery32, PeerSyncReplicationTest, TestSolr4Spatial2, TestReRankQParserPlugin, HLLUtilTest, ExplicitHLLTest, HdfsThreadLeakTest, LeaderFailoverAfterPartitionTest, TestFunctionQuery, UUIDUpdateProcessorFallbackTest, DebugComponentTest, TestExportWriter, DistributedFacetPivotSmallAdvancedTest, HdfsBasicDistributedZkTest]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=HdfsBasicDistributedZkTest -Dtests.seed=AE887A61CD5492CE -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=da -Dtests.timezone=Europe/Oslo -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J0 | HdfsBasicDistributedZkTest (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 3 threads leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsBasicDistributedZkTest:
[junit4] > 1) Thread[id=16924, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HdfsBasicDistributedZkTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 2) Thread[id=16912, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HdfsBasicDistributedZkTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 3) Thread[id=16904, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HdfsBasicDistributedZkTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([AE887A61CD5492CE]:0)
[junit4] Completed [680/730 (3!)] on J0 in 109.92s, 1 test, 2 errors <<< FAILURES!
[...truncated 43473 lines...]
[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 32 - Failure
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/32/
4 tests failed.
FAILED: org.apache.lucene.index.TestIndexSorting.testRandom3
Error Message:
Java heap space
Stack Trace:
java.lang.OutOfMemoryError: Java heap space
at __randomizedtesting.SeedInfo.seed([2E240DD4F2684B9D:8CFC430E969A629B]:0)
at org.apache.lucene.util.packed.Packed64.<init>(Packed64.java:73)
at org.apache.lucene.util.packed.PackedInts.getMutable(PackedInts.java:972)
at org.apache.lucene.util.packed.PackedInts.getMutable(PackedInts.java:939)
at org.apache.lucene.util.packed.GrowableWriter.ensureCapacity(GrowableWriter.java:80)
at org.apache.lucene.util.packed.GrowableWriter.set(GrowableWriter.java:88)
at org.apache.lucene.util.packed.AbstractPagedMutable.set(AbstractPagedMutable.java:98)
at org.apache.lucene.util.fst.NodeHash.addNew(NodeHash.java:152)
at org.apache.lucene.util.fst.NodeHash.rehash(NodeHash.java:169)
at org.apache.lucene.util.fst.NodeHash.add(NodeHash.java:133)
at org.apache.lucene.util.fst.Builder.compileNode(Builder.java:200)
at org.apache.lucene.util.fst.Builder.freezeTail(Builder.java:296)
at org.apache.lucene.util.fst.Builder.add(Builder.java:400)
at org.apache.lucene.codecs.memory.MemoryDocValuesConsumer.writeFST(MemoryDocValuesConsumer.java:373)
at org.apache.lucene.codecs.memory.MemoryDocValuesConsumer.addSortedField(MemoryDocValuesConsumer.java:416)
at org.apache.lucene.codecs.memory.MemoryDocValuesConsumer.addSortedField(MemoryDocValuesConsumer.java:406)
at org.apache.lucene.codecs.DocValuesConsumer.mergeSortedField(DocValuesConsumer.java:527)
at org.apache.lucene.codecs.DocValuesConsumer.merge(DocValuesConsumer.java:139)
at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.merge(PerFieldDocValuesFormat.java:151)
at org.apache.lucene.index.SegmentMerger.mergeDocValues(SegmentMerger.java:181)
at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:125)
at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4392)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4032)
at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2235)
at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:2068)
at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:2019)
at org.apache.lucene.index.TestIndexSorting.testRandom3(TestIndexSorting.java:2318)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
FAILED: org.apache.solr.update.AutoCommitTest.testMaxDocs
Error Message:
Exception during query
Stack Trace:
java.lang.RuntimeException: Exception during query
at __randomizedtesting.SeedInfo.seed([40BFB5BAEEAE1CBC:F93E6365C2441836]:0)
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:880)
at org.apache.solr.update.AutoCommitTest.testMaxDocs(AutoCommitTest.java:225)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
</response>
request was:q=id:14&qt=&start=0&rows=20&version=2.2
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:873)
... 40 more
FAILED: org.apache.solr.update.AutoCommitTest.testMaxTime
Error Message:
Exception during query
Stack Trace:
java.lang.RuntimeException: Exception during query
at __randomizedtesting.SeedInfo.seed([40BFB5BAEEAE1CBC:DA4BC85870348080]:0)
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:880)
at org.apache.solr.update.AutoCommitTest.testMaxTime(AutoCommitTest.java:270)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
</response>
request was:q=id:529&qt=&start=0&rows=20&version=2.2
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:873)
... 40 more
FAILED: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test
Error Message:
expected:<2> but was:<3>
Stack Trace:
java.lang.AssertionError: expected:<2> but was:<3>
at __randomizedtesting.SeedInfo.seed([40BFB5BAEEAE1CBC:C8EB8A6040527144]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.assertSliceAndReplicaCount(SharedFSAutoReplicaFailoverTest.java:403)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.testBasics(SharedFSAutoReplicaFailoverTest.java:248)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test(SharedFSAutoReplicaFailoverTest.java:136)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 1148 lines...]
[junit4] Suite: org.apache.lucene.index.TestIndexSorting
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestIndexSorting -Dtests.method=testRandom3 -Dtests.seed=2E240DD4F2684B9D -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=ru -Dtests.timezone=America/Costa_Rica -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] ERROR 243s J1 | TestIndexSorting.testRandom3 <<<
[junit4] > Throwable #1: java.lang.OutOfMemoryError: Java heap space
[junit4] > at __randomizedtesting.SeedInfo.seed([2E240DD4F2684B9D:8CFC430E969A629B]:0)
[junit4] > at org.apache.lucene.util.packed.Packed64.<init>(Packed64.java:73)
[junit4] > at org.apache.lucene.util.packed.PackedInts.getMutable(PackedInts.java:972)
[junit4] > at org.apache.lucene.util.packed.PackedInts.getMutable(PackedInts.java:939)
[junit4] > at org.apache.lucene.util.packed.GrowableWriter.ensureCapacity(GrowableWriter.java:80)
[junit4] > at org.apache.lucene.util.packed.GrowableWriter.set(GrowableWriter.java:88)
[junit4] > at org.apache.lucene.util.packed.AbstractPagedMutable.set(AbstractPagedMutable.java:98)
[junit4] > at org.apache.lucene.util.fst.NodeHash.addNew(NodeHash.java:152)
[junit4] > at org.apache.lucene.util.fst.NodeHash.rehash(NodeHash.java:169)
[junit4] > at org.apache.lucene.util.fst.NodeHash.add(NodeHash.java:133)
[junit4] > at org.apache.lucene.util.fst.Builder.compileNode(Builder.java:200)
[junit4] > at org.apache.lucene.util.fst.Builder.freezeTail(Builder.java:296)
[junit4] > at org.apache.lucene.util.fst.Builder.add(Builder.java:400)
[junit4] > at org.apache.lucene.codecs.memory.MemoryDocValuesConsumer.writeFST(MemoryDocValuesConsumer.java:373)
[junit4] > at org.apache.lucene.codecs.memory.MemoryDocValuesConsumer.addSortedField(MemoryDocValuesConsumer.java:416)
[junit4] > at org.apache.lucene.codecs.memory.MemoryDocValuesConsumer.addSortedField(MemoryDocValuesConsumer.java:406)
[junit4] > at org.apache.lucene.codecs.DocValuesConsumer.mergeSortedField(DocValuesConsumer.java:527)
[junit4] > at org.apache.lucene.codecs.DocValuesConsumer.merge(DocValuesConsumer.java:139)
[junit4] > at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.merge(PerFieldDocValuesFormat.java:151)
[junit4] > at org.apache.lucene.index.SegmentMerger.mergeDocValues(SegmentMerger.java:181)
[junit4] > at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:125)
[junit4] > at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4392)
[junit4] > at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4032)
[junit4] > at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
[junit4] > at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2235)
[junit4] > at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:2068)
[junit4] > at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:2019)
[junit4] > at org.apache.lucene.index.TestIndexSorting.testRandom3(TestIndexSorting.java:2318)
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/build/core/test/J1/temp/lucene.index.TestIndexSorting_2E240DD4F2684B9D-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {docs=PostingsFormat(name=Direct), sparse_text=PostingsFormat(name=LuceneVarGapFixedInterval), norms=PostingsFormat(name=Direct), positions=PostingsFormat(name=Memory), id=PostingsFormat(name=Direct), term_vectors=PostingsFormat(name=Direct)}, docValues:{multi_valued_long=DocValuesFormat(name=Asserting), foo=DocValuesFormat(name=Asserting), numeric=DocValuesFormat(name=Memory), float=DocValuesFormat(name=Lucene70), multi_valued_numeric=DocValuesFormat(name=Memory), long=DocValuesFormat(name=Lucene70), points=DocValuesFormat(name=Direct), sparse_int=DocValuesFormat(name=Lucene70), bar=DocValuesFormat(name=Memory), multi_valued_double=DocValuesFormat(name=Memory), docs=DocValuesFormat(name=Memory), id=DocValuesFormat(name=Memory), multi_valued_int=DocValuesFormat(name=Direct), multi_valued_bytes=DocValuesFormat(name=Memory), sparse_text=DocValuesFormat(name=Lucene70), double=DocValuesFormat(name=Memory), positions=DocValuesFormat(name=Asserting), sparse_binary=DocValuesFormat(name=Lucene70), int=DocValuesFormat(name=Memory), sorted=DocValuesFormat(name=Memory), multi_valued_string=DocValuesFormat(name=Memory), norms=DocValuesFormat(name=Memory), sparse=DocValuesFormat(name=Lucene70), bytes=DocValuesFormat(name=Memory), binary=DocValuesFormat(name=Memory), dense_int=DocValuesFormat(name=Memory), multi_valued_float=DocValuesFormat(name=Asserting), term_vectors=DocValuesFormat(name=Memory)}, maxPointsInLeafNode=723, maxMBSortInHeap=6.938002432028471, sim=RandomSimilarity(queryNorm=true): {positions=DFR I(ne)1, id=DFR I(F)LZ(0.3), term_vectors=DFR I(ne)2}, locale=ru, timezone=America/Costa_Rica
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=318313096,total=379584512
[junit4] 2> NOTE: All tests run in this JVM: [TestInfoStream, TestParallelTermEnum, TestCodecUtil, TestDocIdsWriter, TestIndexOrDocValuesQuery, TestCustomSearcherSort, TestDuelingCodecs, TestOmitPositions, TestPriorityQueue, TestRecyclingByteBlockAllocator, TestAllFilesDetectTruncation, TestWeakIdentityMap, TestCharArraySet, TestElevationComparator, TestCustomNorms, TestSpanExplanationsOfNonMatches, TestFilterLeafReader, TestPointValues, TestTieredMergePolicy, TestTransactionRollback, TestQueryBuilder, TestRecyclingIntBlockAllocator, TestFieldValueQuery, TestSortRandom, TestDemo, TestSameScoresWithThreads, TestFastDecompressionMode, TestSpanOrQuery, TestLucene60FieldInfoFormat, TestCloseableThreadLocal, TestTermQuery, TestLockFactory, TestMultiTermConstantScore, TestByteSlices, TestSpanFirstQuery, TestPrefixInBooleanQuery, TestDeterminism, TestRAMDirectory, FuzzyTermOnShortTermsTest, Test2BSortedDocValuesFixedSorted, TestLRUQueryCache, TestBufferedChecksum, TestMSBRadixSorter, TestParallelLeafReader, TestSpanExplanations, TestSPIClassIterator, TestPointQueries, TestLucene70DocValuesFormat, TestWildcard, TestDuelingCodecsAtNight, TestBooleanOr, TestPolygon, TestDoubleValuesSource, TestFieldType, TestExternalCodecs, TestForTooMuchCloning, TestOmitTf, TestMinimize, TestDocValuesQueries, TestBooleanQuery, TestBM25Similarity, TestMultiPhraseEnum, TestStressNRT, TestInPlaceMergeSorter, TestTimSorter, TestPrefixRandom, TestTermdocPerf, TestTermVectorsReader, TestDoc, TestIndexWriterOutOfFileDescriptors, TestBlockPostingsFormat3, TestSearcherManager, TestVirtualMethod, TestIndexWriterUnicode, TestIndexSorting]
[junit4] Completed [273/454 (1!)] on J1 in 312.37s, 50 tests, 1 error <<< FAILURES!
[...truncated 644 lines...]
[junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/build/core/test/temp/junit4-J1-20170821_153604_1119205492859019634059.sysout
[junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
[junit4] java.lang.OutOfMemoryError: Java heap space
[junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/heapdumps/java_pid22267.hprof ...
[junit4] Heap dump file created [501544693 bytes in 2.578 secs]
[junit4] <<< JVM J1: EOF ----
[...truncated 9546 lines...]
[junit4] Suite: org.apache.solr.update.AutoCommitTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.update.AutoCommitTest_40BFB5BAEEAE1CBC-001/init-core-data-001
[junit4] 2> 226424 WARN (SUITE-AutoCommitTest-seed#[40BFB5BAEEAE1CBC]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=2 numCloses=2
[junit4] 2> 226424 INFO (SUITE-AutoCommitTest-seed#[40BFB5BAEEAE1CBC]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 226426 INFO (SUITE-AutoCommitTest-seed#[40BFB5BAEEAE1CBC]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 226426 INFO (SUITE-AutoCommitTest-seed#[40BFB5BAEEAE1CBC]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 226427 INFO (SUITE-AutoCommitTest-seed#[40BFB5BAEEAE1CBC]-worker) [ ] o.a.s.SolrTestCaseJ4 ####initCore
[junit4] 2> 226427 INFO (SUITE-AutoCommitTest-seed#[40BFB5BAEEAE1CBC]-worker) [ ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
[junit4] 2> 226456 INFO (SUITE-AutoCommitTest-seed#[40BFB5BAEEAE1CBC]-worker) [ ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 226475 INFO (SUITE-AutoCommitTest-seed#[40BFB5BAEEAE1CBC]-worker) [ ] o.a.s.s.IndexSchema [null] Schema name=test
[junit4] 2> 226620 INFO (SUITE-AutoCommitTest-seed#[40BFB5BAEEAE1CBC]-worker) [ ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 226695 WARN (SUITE-AutoCommitTest-seed#[40BFB5BAEEAE1CBC]-worker) [ ] o.a.s.m.r.SolrJmxReporter No JMX server found. Not exposing Solr metrics via JMX.
[junit4] 2> 226695 WARN (SUITE-AutoCommitTest-seed#[40BFB5BAEEAE1CBC]-worker) [ ] o.a.s.m.r.SolrJmxReporter No JMX server found. Not exposing Solr metrics via JMX.
[junit4] 2> 226695 WARN (SUITE-AutoCommitTest-seed#[40BFB5BAEEAE1CBC]-worker) [ ] o.a.s.m.r.SolrJmxReporter No JMX server found. Not exposing Solr metrics via JMX.
[junit4] 2> 226708 INFO (coreLoadExecutor-107-thread-1) [ x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
[junit4] 2> 226739 INFO (coreLoadExecutor-107-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 226776 INFO (coreLoadExecutor-107-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 227046 INFO (coreLoadExecutor-107-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 227064 INFO (coreLoadExecutor-107-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1, trusted=true
[junit4] 2> 227064 WARN (coreLoadExecutor-107-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter No JMX server found. Not exposing Solr metrics via JMX.
[junit4] 2> 227064 INFO (coreLoadExecutor-107-thread-1) [ x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 227064 INFO (coreLoadExecutor-107-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.update.AutoCommitTest_40BFB5BAEEAE1CBC-001/init-core-data-001/]
[junit4] 2> 227065 INFO (coreLoadExecutor-107-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=28, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6945219450271389]
[junit4] 2> 227147 INFO (coreLoadExecutor-107-thread-1) [ x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 227147 INFO (coreLoadExecutor-107-thread-1) [ x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 227155 INFO (coreLoadExecutor-107-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 227155 INFO (coreLoadExecutor-107-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 227155 INFO (coreLoadExecutor-107-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=43, maxMergeAtOnceExplicit=17, maxMergedSegmentMB=0.9033203125, floorSegmentMB=1.9599609375, forceMergeDeletesPctAllowed=4.359096416575653, segmentsPerTier=17.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.17750527996062795
[junit4] 2> 227156 INFO (coreLoadExecutor-107-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@78828a4a[collection1] main]
[junit4] 2> 227159 INFO (coreLoadExecutor-107-thread-1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
[junit4] 2> 227185 INFO (coreLoadExecutor-107-thread-1) [ x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4] 2> 227889 INFO (coreLoadExecutor-107-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 227889 INFO (coreLoadExecutor-107-thread-1) [ x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1576369626105774080
[junit4] 2> 227890 INFO (searcherExecutor-108-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
[junit4] 2> 227897 INFO (SUITE-AutoCommitTest-seed#[40BFB5BAEEAE1CBC]-worker) [ ] o.a.s.SolrTestCaseJ4 ####initCore end
[junit4] 2> 227898 INFO (searcherExecutor-108-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
[junit4] 2> 227898 INFO (searcherExecutor-108-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
[junit4] 2> 227898 INFO (searcherExecutor-108-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
[junit4] 2> 227899 INFO (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testMaxDocs
[junit4] 2> 227900 INFO (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ ] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
[junit4] 2> 227900 INFO (searcherExecutor-108-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
[junit4] 2> 227901 INFO (searcherExecutor-108-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
[junit4] 2> 227901 INFO (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ ] o.a.s.s.SolrIndexSearcher Opening [Searcher@3481239[collection1] realtime]
[junit4] 2> 227903 INFO (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={_version_=-9223372036854775807&update.distrib=FROMLEADER&versions=true&wt=json&indent=true}{deleteByQuery=*:* (-9223372036854775807)} 0 3
[junit4] 2> 227903 ERROR (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ ] o.a.s.c.CorePropertiesLocator Couldn't load core descriptor from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/core.properties:java.nio.file.NoSuchFileException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/core.properties
[junit4] 2> 227904 INFO (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
[junit4] 2> 227904 INFO (searcherExecutor-108-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
[junit4] 2> 227904 INFO (searcherExecutor-108-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
[junit4] 2> 227905 INFO (searcherExecutor-108-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
[junit4] 2> 227905 INFO (searcherExecutor-108-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@78828a4a[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 227948 INFO (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 228006 INFO (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ ] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 228116 INFO (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 228137 INFO (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ ] o.a.s.c.CoreContainer Reloading SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1
[junit4] 2> 228138 WARN (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.m.r.SolrJmxReporter No JMX server found. Not exposing Solr metrics via JMX.
[junit4] 2> 228138 INFO (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.update.AutoCommitTest_40BFB5BAEEAE1CBC-001/init-core-data-001/]
[junit4] 2> 228174 INFO (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 228174 INFO (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 228176 INFO (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3adb8989[collection1] main]
[junit4] 2> 228176 INFO (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
[junit4] 2> 228177 INFO (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4] 2> 228180 INFO (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 228180 INFO (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1576369626410909696
[junit4] 2> 228188 INFO (searcherExecutor-113-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
[junit4] 2> 228188 INFO (searcherExecutor-113-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
[junit4] 2> 228189 INFO (searcherExecutor-113-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
[junit4] 2> 228189 INFO (searcherExecutor-113-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
[junit4] 2> 228189 INFO (searcherExecutor-113-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
[junit4] 2> 228189 INFO (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=45, maxMergeAtOnceExplicit=12, maxMergedSegmentMB=1.5869140625, floorSegmentMB=1.7705078125, forceMergeDeletesPctAllowed=14.309636822601272, segmentsPerTier=10.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 228190 INFO (searcherExecutor-113-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
[junit4] 2> 228190 INFO (searcherExecutor-113-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
[junit4] 2> 228191 INFO (searcherExecutor-113-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
[junit4] 2> 228191 INFO (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
[junit4] 2> 228191 INFO (searcherExecutor-113-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
[junit4] 2> 228192 INFO (searcherExecutor-113-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@3adb8989[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 228192 INFO (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3432bec9[collection1] main]
[junit4] 2> 228193 INFO (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@552d876d
[junit4] 2> 228193 INFO (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=1429047149
[junit4] 2> 228193 INFO (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3c52887d: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / null
[junit4] 2> 228196 INFO (searcherExecutor-113-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@3432bec9[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 228198 INFO (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[0 (1576369626426638336)]} 0 3
[junit4] 2> 228199 INFO (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[1 (1576369626430832640)]} 0 4
[junit4] 2> 228200 INFO (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[2 (1576369626430832641)]} 0 4
[junit4] 2> 228201 INFO (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[3 (1576369626431881216)]} 0 6
[junit4] 2> 228204 INFO (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[4 (1576369626432929792)]} 0 8
[junit4] 2> 228206 INFO (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[5 (1576369626437124096)]} 0 11
[junit4] 2> 228207 INFO (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[6 (1576369626439221248)]} 0 11
[junit4] 2> 228211 INFO (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[7 (1576369626439221249)]} 0 15
[junit4] 2> 228214 INFO (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[8 (1576369626443415552)]} 0 19
[junit4] 2> 228215 INFO (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[9 (1576369626446561280)]} 0 19
[junit4] 2> 228215 INFO (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[10 (1576369626447609856)]} 0 20
[junit4] 2> 228216 INFO (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[11 (1576369626447609857)]} 0 20
[junit4] 2> 228216 INFO (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[12 (1576369626448658432)]} 0 21
[junit4] 2> 228217 INFO (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[13 (1576369626448658433)]} 0 21
[junit4] 2> 228217 INFO (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=null path=null params={q=id:1&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
[junit4] 2> 228223 INFO (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[14 (1576369626451804160)]} 0 27
[junit4] 2> 228224 INFO (commitScheduler-116-thread-1) [ x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
[junit4] 2> 228225 INFO (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[15 (1576369626457047040)]} 0 29
[junit4] 2> 228226 INFO (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=null path=null params={q=id:14&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
[junit4] 2> 228226 ERROR (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.SolrTestCaseJ4 REQUEST FAILED: xpath=//result[@numFound=1]
[junit4] 2> xml response was: <?xml version="1.0" encoding="UTF-8"?>
[junit4] 2> <response>
[junit4] 2> <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
[junit4] 2> </response>
[junit4] 2>
[junit4] 2> request was:q=id:14&qt=&start=0&rows=20&version=2.2
[junit4] 2> 228226 ERROR (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.SolrTestCaseJ4 REQUEST FAILED: q=id:14&qt=&start=0&rows=20&version=2.2:java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
[junit4] 2> xml response was: <?xml version="1.0" encoding="UTF-8"?>
[junit4] 2> <response>
[junit4] 2> <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
[junit4] 2> </response>
[junit4] 2>
[junit4] 2> request was:q=id:14&qt=&start=0&rows=20&version=2.2
[junit4] 2> at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:873)
[junit4] 2> at org.apache.solr.update.AutoCommitTest.testMaxDocs(AutoCommitTest.java:225)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2>
[junit4] 2> 228227 INFO (TEST-AutoCommitTest.testMaxDocs-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.SolrTestCaseJ4 ###Ending testMaxDocs
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=AutoCommitTest -Dtests.method=testMaxDocs -Dtests.seed=40BFB5BAEEAE1CBC -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=mt-MT -Dtests.timezone=Africa/Maputo -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 0.33s J2 | AutoCommitTest.testMaxDocs <<<
[junit4] > Throwable #1: java.lang.RuntimeException: Exception during query
[junit4] > at __randomizedtesting.SeedInfo.seed([40BFB5BAEEAE1CBC:F93E6365C2441836]:0)
[junit4] > at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:880)
[junit4] > at org.apache.solr.update.AutoCommitTest.testMaxDocs(AutoCommitTest.java:225)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
[junit4] > xml response was: <?xml version="1.0" encoding="UTF-8"?>
[junit4] > <response>
[junit4] > <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
[junit4] > </response>
[junit4] > request was:q=id:14&qt=&start=0&rows=20&version=2.2
[junit4] > at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:873)
[junit4] > ... 40 more
[junit4] 2> 228232 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.SolrTestCaseJ4 ###Starting testCommitWithin
[junit4] 2> 228300 INFO (commitScheduler-116-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@182749fd[collection1] main]
[junit4] 2> 228302 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
[junit4] 2> 228303 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@334d1836[collection1] realtime]
[junit4] 2> 228303 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={_version_=-9223372036854775807&update.distrib=FROMLEADER&versions=true&wt=json&indent=true}{deleteByQuery=*:* (-9223372036854775807)} 0 71
[junit4] 2> 228303 ERROR (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.c.CorePropertiesLocator Couldn't load core descriptor from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/core.properties:java.nio.file.NoSuchFileException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/core.properties
[junit4] 2> 228304 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
[junit4] 2> 228305 INFO (searcherExecutor-113-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@182749fd[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.1.0):C16)))}
[junit4] 2> 228305 INFO (commitScheduler-116-thread-1) [ x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 228335 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 228369 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 228568 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 228578 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.c.CoreContainer Reloading SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1
[junit4] 2> 228578 WARN (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.m.r.SolrJmxReporter No JMX server found. Not exposing Solr metrics via JMX.
[junit4] 2> 228578 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.update.AutoCommitTest_40BFB5BAEEAE1CBC-001/init-core-data-001/]
[junit4] 2> 228708 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 228708 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 228709 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@6f886252[collection1] main]
[junit4] 2> 228709 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
[junit4] 2> 228710 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4] 2> 228713 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 228714 INFO (searcherExecutor-117-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
[junit4] 2> 228715 INFO (searcherExecutor-117-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
[junit4] 2> 228715 INFO (searcherExecutor-117-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
[junit4] 2> 228715 INFO (searcherExecutor-117-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
[junit4] 2> 228717 INFO (searcherExecutor-117-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
[junit4] 2> 228718 INFO (searcherExecutor-117-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
[junit4] 2> 228718 INFO (searcherExecutor-117-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
[junit4] 2> 228718 INFO (searcherExecutor-117-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
[junit4] 2> 228718 INFO (searcherExecutor-117-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
[junit4] 2> 228719 INFO (searcherExecutor-117-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@6f886252[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 228734 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=19, maxMergeAtOnceExplicit=41, maxMergedSegmentMB=74.0341796875, floorSegmentMB=1.8916015625, forceMergeDeletesPctAllowed=10.825225585052507, segmentsPerTier=26.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 228738 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
[junit4] 2> 228739 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@376505d7[collection1] main]
[junit4] 2> 228739 INFO (searcherExecutor-117-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@376505d7[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 228739 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@10370789
[junit4] 2> 228740 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=272041865
[junit4] 2> 228740 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7be6b2b: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / null
[junit4] 2> 228743 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[529 (1576369627000209408)]} 0 2
[junit4] 2> 228744 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=null path=null params={q=id:529&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
[junit4] 2> 232743 INFO (commitScheduler-120-thread-1) [ x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
[junit4] 2> 232760 INFO (commitScheduler-120-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1ee5370b[collection1] main]
[junit4] 2> 232762 INFO (searcherExecutor-117-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1ee5370b[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_1(7.1.0):C1)))}
[junit4] 2> 232762 INFO (commitScheduler-120-thread-1) [ x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 232777 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[530 (1576369631229116416)]} 0 4035
[junit4] 2> 232778 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=null path=null params={q=id:530&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
[junit4] 2> 232779 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {delete=[529 (-1576369631233310720)]} 0 4037
[junit4] 2> 232779 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=null path=null params={q=id:529&qt=&start=0&rows=20&version=2.2} hits=1 status=0 QTime=0
[junit4] 2> 234779 INFO (commitScheduler-120-thread-1) [ x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
[junit4] 2> 234839 INFO (commitScheduler-120-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@d1176f4[collection1] main]
[junit4] 2> 234841 INFO (searcherExecutor-117-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@d1176f4[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_2(7.1.0):C1)))}
[junit4] 2> 234842 INFO (commitScheduler-120-thread-1) [ x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 235034 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=null path=null params={q=id:529&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=1
[junit4] 2> 235035 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=null path=null params={q=id:530&qt=&start=0&rows=20&version=2.2} hits=1 status=0 QTime=0
[junit4] 2> 235038 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500 (1576369633599946752)]} 0 6296
[junit4] 2> 235039 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500 (1576369633602043904)]} 0 6297
[junit4] 2> 235039 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500 (1576369633603092480)]} 0 6298
[junit4] 2> 235040 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500 (1576369633604141056)]} 0 6298
[junit4] 2> 235040 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500 (1576369633604141057)]} 0 6299
[junit4] 2> 235041 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500 (1576369633604141058)]} 0 6299
[junit4] 2> 235041 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500 (1576369633605189632)]} 0 6300
[junit4] 2> 235042 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500 (1576369633605189633)]} 0 6300
[junit4] 2> 235042 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500 (1576369633606238208)]} 0 6300
[junit4] 2> 235043 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[500 (1576369633606238209)]} 0 6301
[junit4] 2> 235043 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=null path=null params={q=id:500&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
[junit4] 2> 235044 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {delete=[530 (-1576369633607286784)]} 0 6302
[junit4] 2> 235044 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {delete=[530 (-1576369633608335360)]} 0 6302
[junit4] 2> 235044 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {delete=[530 (-1576369633608335361)]} 0 6302
[junit4] 2> 235044 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {delete=[530 (-1576369633608335362)]} 0 6302
[junit4] 2> 235044 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {delete=[530 (-1576369633608335363)]} 0 6303
[junit4] 2> 235045 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {delete=[530 (-1576369633609383936)]} 0 6303
[junit4] 2> 235045 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {delete=[530 (-1576369633609383937)]} 0 6303
[junit4] 2> 235045 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {delete=[530 (-1576369633609383938)]} 0 6303
[junit4] 2> 235045 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {delete=[530 (-1576369633609383939)]} 0 6304
[junit4] 2> 235046 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {delete=[530 (-1576369633609383940)]} 0 6304
[junit4] 2> 235046 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=null path=null params={q=id:530&qt=&start=0&rows=20&version=2.2} hits=1 status=0 QTime=0
[junit4] 2> 236044 INFO (commitScheduler-120-thread-1) [ x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
[junit4] 2> 236051 INFO (commitScheduler-120-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@39304f1d[collection1] main]
[junit4] 2> 236052 INFO (searcherExecutor-117-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@39304f1d[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_3(7.1.0):C10/9:delGen=1)))}
[junit4] 2> 236054 INFO (commitScheduler-120-thread-1) [ x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 236299 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=null path=null params={q=id:500&qt=&start=0&rows=20&version=2.2} hits=1 status=0 QTime=0
[junit4] 2> 236301 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=null path=null params={q=id:530&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
[junit4] 2> 236302 INFO (TEST-AutoCommitTest.testCommitWithin-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.SolrTestCaseJ4 ###Ending testCommitWithin
[junit4] 2> 236304 INFO (TEST-AutoCommitTest.testMaxTime-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.SolrTestCaseJ4 ###Starting testMaxTime
[junit4] 2> 236305 INFO (TEST-AutoCommitTest.testMaxTime-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
[junit4] 2> 236306 INFO (TEST-AutoCommitTest.testMaxTime-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1c9ad35d[collection1] realtime]
[junit4] 2> 236306 INFO (TEST-AutoCommitTest.testMaxTime-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={_version_=-9223372036854775807&update.distrib=FROMLEADER&versions=true&wt=json&indent=true}{deleteByQuery=*:* (-9223372036854775807)} 0 1
[junit4] 2> 236307 ERROR (TEST-AutoCommitTest.testMaxTime-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.c.CorePropertiesLocator Couldn't load core descriptor from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/core.properties:java.nio.file.NoSuchFileException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/core.properties
[junit4] 2> 236307 INFO (TEST-AutoCommitTest.testMaxTime-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
[junit4] 2> 236341 INFO (TEST-AutoCommitTest.testMaxTime-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 236377 INFO (TEST-AutoCommitTest.testMaxTime-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 236501 INFO (TEST-AutoCommitTest.testMaxTime-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 236517 INFO (TEST-AutoCommitTest.testMaxTime-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.c.CoreContainer Reloading SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1
[junit4] 2> 236518 WARN (TEST-AutoCommitTest.testMaxTime-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.m.r.SolrJmxReporter No JMX server found. Not exposing Solr metrics via JMX.
[junit4] 2> 236518 INFO (TEST-AutoCommitTest.testMaxTime-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.update.AutoCommitTest_40BFB5BAEEAE1CBC-001/init-core-data-001/]
[junit4] 2> 236577 INFO (TEST-AutoCommitTest.testMaxTime-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 236577 INFO (TEST-AutoCommitTest.testMaxTime-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 236578 INFO (TEST-AutoCommitTest.testMaxTime-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@556c3f06[collection1] main]
[junit4] 2> 236579 INFO (TEST-AutoCommitTest.testMaxTime-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
[junit4] 2> 236579 INFO (TEST-AutoCommitTest.testMaxTime-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4] 2> 236582 INFO (TEST-AutoCommitTest.testMaxTime-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 236588 INFO (searcherExecutor-121-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
[junit4] 2> 236589 INFO (searcherExecutor-121-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
[junit4] 2> 236589 INFO (searcherExecutor-121-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
[junit4] 2> 236589 INFO (searcherExecutor-121-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
[junit4] 2> 236589 INFO (searcherExecutor-121-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
[junit4] 2> 236590 INFO (TEST-AutoCommitTest.testMaxTime-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=31, maxMergeAtOnceExplicit=48, maxMergedSegmentMB=52.958984375, floorSegmentMB=1.7470703125, forceMergeDeletesPctAllowed=23.403136836039675, segmentsPerTier=21.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 236599 INFO (TEST-AutoCommitTest.testMaxTime-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
[junit4] 2> 236599 INFO (searcherExecutor-121-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
[junit4] 2> 236599 INFO (searcherExecutor-121-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
[junit4] 2> 236602 INFO (searcherExecutor-121-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
[junit4] 2> 236602 INFO (searcherExecutor-121-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
[junit4] 2> 236603 INFO (searcherExecutor-121-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@556c3f06[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 236603 INFO (TEST-AutoCommitTest.testMaxTime-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@6ca889a2[collection1] main]
[junit4] 2> 236603 INFO (TEST-AutoCommitTest.testMaxTime-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@2b14c801
[junit4] 2> 236603 INFO (TEST-AutoCommitTest.testMaxTime-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=722782209
[junit4] 2> 236603 INFO (TEST-AutoCommitTest.testMaxTime-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2ce53525: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / null
[junit4] 2> 236607 INFO (searcherExecutor-121-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@6ca889a2[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 236620 INFO (TEST-AutoCommitTest.testMaxTime-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[529 (1576369635245162496)]} 0 15
[junit4] 2> 236620 INFO (TEST-AutoCommitTest.testMaxTime-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=null path=null params={q=id:529&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
[junit4] 2> 236621 INFO (TEST-AutoCommitTest.testMaxTime-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[530 (1576369635261939712)]} 0 16
[junit4] 2> 236622 INFO (TEST-AutoCommitTest.testMaxTime-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=null path=null params={q=id:529&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
[junit4] 2> 236622 ERROR (TEST-AutoCommitTest.testMaxTime-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.SolrTestCaseJ4 REQUEST FAILED: xpath=//result[@numFound=1]
[junit4] 2> xml response was: <?xml version="1.0" encoding="UTF-8"?>
[junit4] 2> <response>
[junit4] 2> <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
[junit4] 2> </response>
[junit4] 2>
[junit4] 2> request was:q=id:529&qt=&start=0&rows=20&version=2.2
[junit4] 2> 236622 ERROR (TEST-AutoCommitTest.testMaxTime-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.SolrTestCaseJ4 REQUEST FAILED: q=id:529&qt=&start=0&rows=20&version=2.2:java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
[junit4] 2> xml response was: <?xml version="1.0" encoding="UTF-8"?>
[junit4] 2> <response>
[junit4] 2> <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
[junit4] 2> </response>
[junit4] 2>
[junit4] 2> request was:q=id:529&qt=&start=0&rows=20&version=2.2
[junit4] 2> at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:873)
[junit4] 2> at org.apache.solr.update.AutoCommitTest.testMaxTime(AutoCommitTest.java:270)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2>
[junit4] 2> 236623 INFO (TEST-AutoCommitTest.testMaxTime-seed#[40BFB5BAEEAE1CBC]) [ x:collection1] o.a.s.SolrTestCaseJ4 ###Ending testMaxTime
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=AutoCommitTest -Dtests.method=testMaxTime -Dtests.seed=40BFB5BAEEAE1CBC -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=mt-MT -Dtests.timezone=Africa/Maputo -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 0.32s J2 | AutoCommitTest.testMaxTime <<<
[junit4] > Throwable #1: java.lang.RuntimeException: Exception during query
[junit4] > at __randomizedtesting.SeedInfo.seed([40BFB5BAEEAE1CBC:DA4BC85870348080]:0)
[junit4] > at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:880)
[junit4] > at org.apache.solr.update.AutoCommitTest.testMaxTime(AutoCommitTest.java:270)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
[junit4] > xml response was: <?xml version="1.0" encoding="UTF-8"?>
[junit4] > <response>
[junit4] > <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
[junit4] > </response>
[junit4] > request was:q=id:529&qt=&start=0&rows=20&version=2.2
[junit4] > at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:873)
[junit4] > ... 40 more
[junit4] 2> 236623 INFO (SUITE-AutoCommitTest-seed#[40BFB5BAEEAE1CBC]-worker) [ x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> 236623 INFO (SUITE-AutoCommitTest-seed#[40BFB5BAEEAE1CBC]-worker) [ x:collection1] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1785408474
[junit4] 2> 236623 INFO (SUITE-AutoCommitTest-seed#[40BFB5BAEEAE1CBC]-worker) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 236623 INFO (SUITE-AutoCommitTest-seed#[40BFB5BAEEAE1CBC]-worker) [ x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1b7ad5e5: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / null
[junit4] 2> 236624 INFO (SUITE-AutoCommitTest-seed#[40BFB5BAEEAE1CBC]-worker) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 236624 INFO (SUITE-AutoCommitTest-seed#[40BFB5BAEEAE1CBC]-worker) [ x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@351b5403: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / null
[junit4] 2> 236624 INFO (SUITE-AutoCommitTest-seed#[40BFB5BAEEAE1CBC]-worker) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 236624 INFO (SUITE-AutoCommitTest-seed#[40BFB5BAEEAE1CBC]-worker) [ x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@45d59891: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / null
[junit4] 2> 236624 INFO (SUITE-AutoCommitTest-seed#[40BFB5BAEEAE1CBC]-worker) [ x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 236632 INFO (coreCloseExecutor-125-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@368ca887
[junit4] 2> 236632 INFO (cor
[...truncated too long message...]
[junit4] 2> 13 /solr/collections
[junit4] 2> 8 /solr/live_nodes
[junit4] 2> 5 /solr/overseer/queue
[junit4] 2> 5 /solr/overseer/collection-queue-work
[junit4] 2> 5 /solr/overseer/queue-work
[junit4] 2>
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=SharedFSAutoReplicaFailoverTest -Dtests.method=test -Dtests.seed=40BFB5BAEEAE1CBC -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=be -Dtests.timezone=Pacific/Kiritimati -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] FAILURE 82.0s J2 | SharedFSAutoReplicaFailoverTest.test <<<
[junit4] > Throwable #1: java.lang.AssertionError: expected:<2> but was:<3>
[junit4] > at __randomizedtesting.SeedInfo.seed([40BFB5BAEEAE1CBC:C8EB8A6040527144]:0)
[junit4] > at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.assertSliceAndReplicaCount(SharedFSAutoReplicaFailoverTest.java:403)
[junit4] > at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.testBasics(SharedFSAutoReplicaFailoverTest.java:248)
[junit4] > at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test(SharedFSAutoReplicaFailoverTest.java:136)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 3686378 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[40BFB5BAEEAE1CBC]-worker) [ ] o.a.h.h.s.d.DirectoryScanner DirectoryScanner: shutdown has been called
[junit4] 2> 3686408 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[40BFB5BAEEAE1CBC]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
[junit4] 2> 3686509 WARN (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_40BFB5BAEEAE1CBC-001/tempDir-001/hdfsBaseDir/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_40BFB5BAEEAE1CBC-001/tempDir-001/hdfsBaseDir/data/data4/]] heartbeating to localhost/127.0.0.1:50152) [ ] o.a.h.h.s.d.IncrementalBlockReportManager IncrementalBlockReportManager interrupted
[junit4] 2> 3686509 WARN (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_40BFB5BAEEAE1CBC-001/tempDir-001/hdfsBaseDir/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_40BFB5BAEEAE1CBC-001/tempDir-001/hdfsBaseDir/data/data4/]] heartbeating to localhost/127.0.0.1:50152) [ ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-38432504-10.41.0.5-1503346599670 (Datanode Uuid 433cbaf4-1043-4d5a-8a2f-5339251af270) service to localhost/127.0.0.1:50152
[junit4] 2> 3686512 WARN (SUITE-SharedFSAutoReplicaFailoverTest-seed#[40BFB5BAEEAE1CBC]-worker) [ ] o.a.h.h.s.d.DirectoryScanner DirectoryScanner: shutdown has been called
[junit4] 2> 3686534 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[40BFB5BAEEAE1CBC]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
[junit4] 2> 3686540 WARN (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_40BFB5BAEEAE1CBC-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_40BFB5BAEEAE1CBC-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to localhost/127.0.0.1:50152) [ ] o.a.h.h.s.d.IncrementalBlockReportManager IncrementalBlockReportManager interrupted
[junit4] 2> 3686540 WARN (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_40BFB5BAEEAE1CBC-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_40BFB5BAEEAE1CBC-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to localhost/127.0.0.1:50152) [ ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-38432504-10.41.0.5-1503346599670 (Datanode Uuid 2cdc0506-3a40-41ac-83e5-0edd11ce48e0) service to localhost/127.0.0.1:50152
[junit4] 2> 3686547 INFO (SUITE-SharedFSAutoReplicaFailoverTest-seed#[40BFB5BAEEAE1CBC]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_40BFB5BAEEAE1CBC-001
[junit4] 2> Aug 21, 2017 8:18:04 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 34 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {id=PostingsFormat(name=MockRandom), text=PostingsFormat(name=Memory)}, docValues:{_version_=DocValuesFormat(name=Lucene70)}, maxPointsInLeafNode=1410, maxMBSortInHeap=6.979102404526774, sim=RandomSimilarity(queryNorm=true): {}, locale=be, timezone=Pacific/Kiritimati
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=2,free=187101032,total=522715136
[junit4] 2> NOTE: All tests run in this JVM: [URLClassifyProcessorTest, SortByFunctionTest, TestLFUCache, PolyFieldTest, TestIndexSearcher, TestObjectReleaseTracker, TestTolerantSearch, QueryEqualityTest, AutoCommitTest, SuggesterTest, TestHttpShardHandlerFactory, TestSQLHandlerNonCloud, RequestLoggingTest, TestSort, TestRequestForwarding, TestDynamicLoading, OutOfBoxZkACLAndCredentialsProvidersTest, TemplateUpdateProcessorTest, TestPKIAuthenticationPlugin, TestExportWriter, FacetPivotSmallTest, SuggesterFSTTest, UniqFieldsUpdateProcessorFactoryTest, LeaderElectionTest, HardAutoCommitTest, TestSolrCoreProperties, DeleteNodeTest, MoveReplicaTest, HdfsWriteToMultipleCollectionsTest, TestRecovery, BasicDistributedZk2Test, SimplePostToolTest, TestSystemCollAutoCreate, DirectUpdateHandlerTest, DistributedFacetPivotSmallAdvancedTest, TestInitParams, TestInitQParser, SolrTestCaseJ4Test, CloneFieldUpdateProcessorFactoryTest, PingRequestHandlerTest, CoreSorterTest, NotRequiredUniqueKeyTest, RecoveryZkTest, SharedFSAutoReplicaFailoverUtilsTest, TestStressReorder, TestDefaultStatsCache, TestFoldingMultitermQuery, DirectUpdateHandlerOptimizeTest, TestConfigSets, NumericFieldsTest, TestStandardQParsers, TestDynamicFieldResource, UUIDUpdateProcessorFallbackTest, TestCloudNestedDocsSort, TestFieldTypeCollectionResource, TestQuerySenderListener, ParsingFieldUpdateProcessorsTest, PrimitiveFieldTypeTest, TestApiFramework, TestTlogReplica, DocumentBuilderTest, FastVectorHighlighterTest, TestHdfsCloudBackupRestore, AddSchemaFieldsUpdateProcessorFactoryTest, SolrJmxReporterCloudTest, CdcrReplicationDistributedZkTest, SolrCloudReportersTest, TestDistributedMissingSort, TestConfigReload, TermVectorComponentDistributedTest, TestSortByMinMaxFunction, TestFastWriter, TestDistribDocBasedVersion, FullSolrCloudDistribCmdsTest, TestCodecSupport, DocValuesNotIndexedTest, TestBulkSchemaAPI, ShufflingReplicaListTransformerTest, InfoHandlerTest, TestExactStatsCache, TestScoreJoinQPScore, HLLUtilTest, TestManagedSchema, RemoteQueryErrorTest, TestJsonFacets, LeaderInitiatedRecoveryOnCommitTest, TestManagedResource, TestLMJelinekMercerSimilarityFactory, MissingSegmentRecoveryTest, TestSerializedLuceneMatchVersion, SolrInfoBeanTest, TestSubQueryTransformer, OverseerTaskQueueTest, DateFieldTest, TestJettySolrRunner, SolrCoreMetricManagerTest, TestSolrXml, TestExceedMaxTermLength, TestAddFieldRealTimeGet, XsltUpdateRequestHandlerTest, SpatialRPTFieldTypeTest, DeleteStatusTest, PathHierarchyTokenizerFactoryTest, BasicAuthStandaloneTest, TestConfigOverlay, TestUseDocValuesAsStored, TestRawTransformer, TestDFRSimilarityFactory, MergeStrategyTest, CollectionsAPIAsyncDistributedZkTest, PeerSyncWithIndexFingerprintCachingTest, DistributedQueryComponentCustomSortTest, TlogReplayBufferedWhileIndexingTest, ClusterStateTest, MinimalSchemaTest, CdcrVersionReplicationTest, TestFieldCacheWithThreads, TestSchemaResource, BasicDistributedZkTest, SharedFSAutoReplicaFailoverTest]
[junit4] Completed [522/729 (2!)] on J2 in 94.47s, 1 test, 1 failure <<< FAILURES!
[...truncated 8261 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/build.xml:818: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/build.xml:762: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid22267.hprof
Total time: 326 minutes 2 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 31 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/31/
6 tests failed.
FAILED: org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([799D43D729FE9D6]:0)
FAILED: junit.framework.TestSuite.org.apache.lucene.spatial3d.TestGeo3DPoint
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([799D43D729FE9D6]:0)
FAILED: org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test
Error Message:
Expected 2 of 3 replicas to be active but only found 1; [core_node3:{"core":"c8n_1x3_lf_shard1_replica_n1","base_url":"http://127.0.0.1:40499","node_name":"127.0.0.1:40499_","state":"active","type":"NRT","leader":"true"}]; clusterState: DocCollection(c8n_1x3_lf//collections/c8n_1x3_lf/state.json/18)={ "pullReplicas":"0", "replicationFactor":"1", "shards":{"shard1":{ "range":"80000000-7fffffff", "state":"active", "replicas":{ "core_node3":{ "core":"c8n_1x3_lf_shard1_replica_n1", "base_url":"http://127.0.0.1:40499", "node_name":"127.0.0.1:40499_", "state":"active", "type":"NRT", "leader":"true"}, "core_node5":{ "core":"c8n_1x3_lf_shard1_replica_n2", "base_url":"http://127.0.0.1:35915", "node_name":"127.0.0.1:35915_", "state":"down", "type":"NRT"}, "core_node6":{ "state":"down", "base_url":"http://127.0.0.1:33140", "core":"c8n_1x3_lf_shard1_replica_n4", "node_name":"127.0.0.1:33140_", "type":"NRT"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"false", "nrtReplicas":"3", "tlogReplicas":"0"}
Stack Trace:
java.lang.AssertionError: Expected 2 of 3 replicas to be active but only found 1; [core_node3:{"core":"c8n_1x3_lf_shard1_replica_n1","base_url":"http://127.0.0.1:40499","node_name":"127.0.0.1:40499_","state":"active","type":"NRT","leader":"true"}]; clusterState: DocCollection(c8n_1x3_lf//collections/c8n_1x3_lf/state.json/18)={
"pullReplicas":"0",
"replicationFactor":"1",
"shards":{"shard1":{
"range":"80000000-7fffffff",
"state":"active",
"replicas":{
"core_node3":{
"core":"c8n_1x3_lf_shard1_replica_n1",
"base_url":"http://127.0.0.1:40499",
"node_name":"127.0.0.1:40499_",
"state":"active",
"type":"NRT",
"leader":"true"},
"core_node5":{
"core":"c8n_1x3_lf_shard1_replica_n2",
"base_url":"http://127.0.0.1:35915",
"node_name":"127.0.0.1:35915_",
"state":"down",
"type":"NRT"},
"core_node6":{
"state":"down",
"base_url":"http://127.0.0.1:33140",
"core":"c8n_1x3_lf_shard1_replica_n4",
"node_name":"127.0.0.1:33140_",
"type":"NRT"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"3",
"tlogReplicas":"0"}
at __randomizedtesting.SeedInfo.seed([A552ECE1FD218366:2D06D33B53DDEE9E]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.testRf3WithLeaderFailover(LeaderFailoverAfterPartitionTest.java:169)
at org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test(LeaderFailoverAfterPartitionTest.java:56)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.ShardSplitTest.testSplitAfterFailedSplit
Error Message:
expected:<1> but was:<2>
Stack Trace:
java.lang.AssertionError: expected:<1> but was:<2>
at __randomizedtesting.SeedInfo.seed([A552ECE1FD218366:5C1F7F4EC154CEEC]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.ShardSplitTest.testSplitAfterFailedSplit(ShardSplitTest.java:279)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.HttpPartitionTest.test
Error Message:
Didn't see all replicas for shard shard1 in c8n_1x2 come up within 90000 ms! ClusterState: { "collMinRf_1x3":{ "pullReplicas":"0", "replicationFactor":"1", "shards":{"shard1":{ "range":"80000000-7fffffff", "state":"active", "replicas":{ "core_node4":{ "core":"collMinRf_1x3_shard1_replica_n2", "base_url":"http://127.0.0.1:46751", "node_name":"127.0.0.1:46751_", "state":"active", "type":"NRT"}, "core_node5":{ "core":"collMinRf_1x3_shard1_replica_n1", "base_url":"http://127.0.0.1:46787", "node_name":"127.0.0.1:46787_", "state":"active", "type":"NRT"}, "core_node6":{ "core":"collMinRf_1x3_shard1_replica_n3", "base_url":"http://127.0.0.1:46044", "node_name":"127.0.0.1:46044_", "state":"active", "type":"NRT", "leader":"true"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"false", "nrtReplicas":"3", "tlogReplicas":"0"}, "collection1":{ "pullReplicas":"0", "replicationFactor":"1", "shards":{ "shard1":{ "range":"80000000-ffffffff", "state":"active", "replicas":{"core_node44":{ "core":"collection1_shard1_replica_n43", "base_url":"http://127.0.0.1:46044", "node_name":"127.0.0.1:46044_", "state":"active", "type":"NRT", "leader":"true"}}}, "shard2":{ "range":"0-7fffffff", "state":"active", "replicas":{ "core_node42":{ "core":"collection1_shard2_replica_n41", "base_url":"http://127.0.0.1:46787", "node_name":"127.0.0.1:46787_", "state":"active", "type":"NRT", "leader":"true"}, "core_node46":{ "core":"collection1_shard2_replica_n45", "base_url":"http://127.0.0.1:35126", "node_name":"127.0.0.1:35126_", "state":"active", "type":"NRT"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"false", "nrtReplicas":"1", "tlogReplicas":"0"}, "control_collection":{ "pullReplicas":"0", "replicationFactor":"1", "shards":{"shard1":{ "range":"80000000-7fffffff", "state":"active", "replicas":{"core_node2":{ "core":"control_collection_shard1_replica_n1", "base_url":"http://127.0.0.1:46751", "node_name":"127.0.0.1:46751_", "state":"active", "type":"NRT", "leader":"true"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"false", "nrtReplicas":"1", "tlogReplicas":"0"}, "c8n_1x2":{ "pullReplicas":"0", "replicationFactor":"1", "shards":{"shard1":{ "range":"80000000-7fffffff", "state":"active", "replicas":{ "core_node3":{ "state":"down", "base_url":"http://127.0.0.1:46787", "core":"c8n_1x2_shard1_replica_n1", "node_name":"127.0.0.1:46787_", "type":"NRT"}, "core_node4":{ "core":"c8n_1x2_shard1_replica_n2", "base_url":"http://127.0.0.1:46751", "node_name":"127.0.0.1:46751_", "state":"active", "type":"NRT", "leader":"true"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"false", "nrtReplicas":"2", "tlogReplicas":"0"}}
Stack Trace:
java.lang.AssertionError: Didn't see all replicas for shard shard1 in c8n_1x2 come up within 90000 ms! ClusterState: {
"collMinRf_1x3":{
"pullReplicas":"0",
"replicationFactor":"1",
"shards":{"shard1":{
"range":"80000000-7fffffff",
"state":"active",
"replicas":{
"core_node4":{
"core":"collMinRf_1x3_shard1_replica_n2",
"base_url":"http://127.0.0.1:46751",
"node_name":"127.0.0.1:46751_",
"state":"active",
"type":"NRT"},
"core_node5":{
"core":"collMinRf_1x3_shard1_replica_n1",
"base_url":"http://127.0.0.1:46787",
"node_name":"127.0.0.1:46787_",
"state":"active",
"type":"NRT"},
"core_node6":{
"core":"collMinRf_1x3_shard1_replica_n3",
"base_url":"http://127.0.0.1:46044",
"node_name":"127.0.0.1:46044_",
"state":"active",
"type":"NRT",
"leader":"true"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"3",
"tlogReplicas":"0"},
"collection1":{
"pullReplicas":"0",
"replicationFactor":"1",
"shards":{
"shard1":{
"range":"80000000-ffffffff",
"state":"active",
"replicas":{"core_node44":{
"core":"collection1_shard1_replica_n43",
"base_url":"http://127.0.0.1:46044",
"node_name":"127.0.0.1:46044_",
"state":"active",
"type":"NRT",
"leader":"true"}}},
"shard2":{
"range":"0-7fffffff",
"state":"active",
"replicas":{
"core_node42":{
"core":"collection1_shard2_replica_n41",
"base_url":"http://127.0.0.1:46787",
"node_name":"127.0.0.1:46787_",
"state":"active",
"type":"NRT",
"leader":"true"},
"core_node46":{
"core":"collection1_shard2_replica_n45",
"base_url":"http://127.0.0.1:35126",
"node_name":"127.0.0.1:35126_",
"state":"active",
"type":"NRT"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"1",
"tlogReplicas":"0"},
"control_collection":{
"pullReplicas":"0",
"replicationFactor":"1",
"shards":{"shard1":{
"range":"80000000-7fffffff",
"state":"active",
"replicas":{"core_node2":{
"core":"control_collection_shard1_replica_n1",
"base_url":"http://127.0.0.1:46751",
"node_name":"127.0.0.1:46751_",
"state":"active",
"type":"NRT",
"leader":"true"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"1",
"tlogReplicas":"0"},
"c8n_1x2":{
"pullReplicas":"0",
"replicationFactor":"1",
"shards":{"shard1":{
"range":"80000000-7fffffff",
"state":"active",
"replicas":{
"core_node3":{
"state":"down",
"base_url":"http://127.0.0.1:46787",
"core":"c8n_1x2_shard1_replica_n1",
"node_name":"127.0.0.1:46787_",
"type":"NRT"},
"core_node4":{
"core":"c8n_1x2_shard1_replica_n2",
"base_url":"http://127.0.0.1:46751",
"node_name":"127.0.0.1:46751_",
"state":"active",
"type":"NRT",
"leader":"true"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"2",
"tlogReplicas":"0"}}
at __randomizedtesting.SeedInfo.seed([A552ECE1FD218366:2D06D33B53DDEE9E]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.ensureAllReplicasAreActive(AbstractFullDistribZkTestBase.java:1996)
at org.apache.solr.cloud.HttpPartitionTest.testRf2(HttpPartitionTest.java:368)
at org.apache.solr.cloud.HttpPartitionTest.test(HttpPartitionTest.java:132)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test
Error Message:
Timeout waiting for all live and active
Stack Trace:
java.lang.AssertionError: Timeout waiting for all live and active
at __randomizedtesting.SeedInfo.seed([A552ECE1FD218366:2D06D33B53DDEE9E]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.testBasics(SharedFSAutoReplicaFailoverTest.java:246)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test(SharedFSAutoReplicaFailoverTest.java:136)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 10155 lines...]
[junit4] Suite: org.apache.lucene.spatial3d.TestGeo3DPoint
[junit4] 2> Rgp 19, 2017 7:49:01 PM com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
[junit4] 2> WARNING: Suite execution timed out: org.apache.lucene.spatial3d.TestGeo3DPoint
[junit4] 2> 1) Thread[id=1, name=main, state=WAITING, group=main]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Thread.join(Thread.java:1252)
[junit4] 2> at java.lang.Thread.join(Thread.java:1326)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:608)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:457)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:251)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:368)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:13)
[junit4] 2> 2) Thread[id=14, name=TEST-TestGeo3DPoint.testRandomBig-seed#[799D43D729FE9D6], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at org.apache.lucene.spatial3d.geom.Plane.addPoint(Plane.java:1953)
[junit4] 2> at org.apache.lucene.spatial3d.geom.Plane.recordBounds(Plane.java:1418)
[junit4] 2> at org.apache.lucene.spatial3d.geom.XYZBounds.addPlane(XYZBounds.java:161)
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoStandardPath$PathSegment.getBounds(GeoStandardPath.java:848)
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoStandardPath.getBounds(GeoStandardPath.java:315)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.verify(TestGeo3DPoint.java:846)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.doTestRandom(TestGeo3DPoint.java:548)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig(TestGeo3DPoint.java:480)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 3) Thread[id=13, name=SUITE-TestGeo3DPoint-seed#[799D43D729FE9D6], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at java.lang.Thread.getStackTrace(Thread.java:1559)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:696)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:693)
[junit4] 2> at java.security.AccessController.doPrivileged(Native Method)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getStackTrace(ThreadLeakControl.java:693)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getThreadsWithTraces(ThreadLeakControl.java:709)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:689)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:65)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:415)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:678)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:140)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:598)
[junit4] 2> 4) Thread[id=9, name=JUnit4-serializer-daemon, state=TIMED_WAITING, group=main]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:50)
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestGeo3DPoint -Dtests.method=testRandomBig -Dtests.seed=799D43D729FE9D6 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=lt-LT -Dtests.timezone=Europe/Brussels -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 6534s J1 | TestGeo3DPoint.testRandomBig <<<
[junit4] > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
[junit4] > at __randomizedtesting.SeedInfo.seed([799D43D729FE9D6]:0)
[junit4] 2> Rgp 19, 2017 7:49:02 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> Rgp 19, 2017 7:49:22 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.lucene.spatial3d.TestGeo3DPoint:
[junit4] 2> 1) Thread[id=14, name=TEST-TestGeo3DPoint.testRandomBig-seed#[799D43D729FE9D6], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at org.apache.lucene.spatial3d.geom.PlanetModel.pointOnSurface(PlanetModel.java:159)
[junit4] 2> at org.apache.lucene.spatial3d.geom.Plane.recordBounds(Plane.java:1253)
[junit4] 2> at org.apache.lucene.spatial3d.geom.XYZBounds.addPlane(XYZBounds.java:161)
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoStandardPath$PathSegment.getBounds(GeoStandardPath.java:847)
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoStandardPath.getBounds(GeoStandardPath.java:315)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.verify(TestGeo3DPoint.java:846)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.doTestRandom(TestGeo3DPoint.java:548)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig(TestGeo3DPoint.java:480)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Rgp 19, 2017 7:49:22 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=14, name=TEST-TestGeo3DPoint.testRandomBig-seed#[799D43D729FE9D6], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/build/spatial3d/test/J1/temp/lucene.spatial3d.TestGeo3DPoint_799D43D729FE9D6-001
[junit4] 2> Aug 19, 2017 5:49:22 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: All leaked threads terminated.
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {id=PostingsFormat(name=Memory)}, docValues:{id=DocValuesFormat(name=Lucene70), point=DocValuesFormat(name=Lucene70)}, maxPointsInLeafNode=811, maxMBSortInHeap=6.952541407723767, sim=RandomSimilarity(queryNorm=false): {}, locale=lt-LT, timezone=Europe/Brussels
[junit4] 2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=134771576,total=456130560
[junit4] 2> NOTE: All tests run in this JVM: [XYZSolidTest, TestGeo3DPoint]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestGeo3DPoint -Dtests.seed=799D43D729FE9D6 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=lt-LT -Dtests.timezone=Europe/Brussels -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J1 | TestGeo3DPoint (suite) <<<
[junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4] > at __randomizedtesting.SeedInfo.seed([799D43D729FE9D6]:0)
[junit4] Completed [13/14 (1!)] on J1 in 7220.66s, 10 tests, 2 errors <<< FAILURES!
[...truncated 2618 lines...]
[junit4] Suite: org.apache.solr.cloud.ShardSplitTest
[junit4] 2> 3605143 INFO (SUITE-ShardSplitTest-seed#[A552ECE1FD218366]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ShardSplitTest_A552ECE1FD218366-001/init-core-data-001
[junit4] 2> 3605143 WARN (SUITE-ShardSplitTest-seed#[A552ECE1FD218366]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=32 numCloses=32
[junit4] 2> 3605143 INFO (SUITE-ShardSplitTest-seed#[A552ECE1FD218366]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 3605238 INFO (SUITE-ShardSplitTest-seed#[A552ECE1FD218366]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
[junit4] 2> 3605238 INFO (SUITE-ShardSplitTest-seed#[A552ECE1FD218366]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /kg_o/c
[junit4] 2> 3605250 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 3605282 INFO (Thread-4708) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 3605282 INFO (Thread-4708) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 3605319 ERROR (Thread-4708) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 3605382 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.c.ZkTestServer start zk server on port:37873
[junit4] 2> 3605505 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 3605518 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 3605521 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 3605534 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 3605535 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 3605536 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 3605537 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 3605550 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 3605551 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 3605552 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 3605553 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 3605583 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
[junit4] 2> 3605599 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15dfbdbefcb0001, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 3605993 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 3605996 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7f54d3d9{/kg_o/c,null,AVAILABLE}
[junit4] 2> 3606111 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@29ab43c8{HTTP/1.1,[http/1.1]}{127.0.0.1:46719}
[junit4] 2> 3606111 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.e.j.s.Server Started @3618070ms
[junit4] 2> 3606111 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ShardSplitTest_A552ECE1FD218366-001/tempDir-001/control/data, replicaType=NRT, hostContext=/kg_o/c, hostPort=46719, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ShardSplitTest_A552ECE1FD218366-001/control-001/cores}
[junit4] 2> 3606111 ERROR (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3606111 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
[junit4] 2> 3606111 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3606112 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 3606112 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-19T18:58:02.120Z
[junit4] 2> 3606144 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 3606144 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ShardSplitTest_A552ECE1FD218366-001/control-001/solr.xml
[junit4] 2> 3606177 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@2b064754, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 3606209 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37873/solr
[junit4] 2> 3606839 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [n:127.0.0.1:46719_kg_o%2Fc ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3606840 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [n:127.0.0.1:46719_kg_o%2Fc ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:46719_kg_o%2Fc
[junit4] 2> 3606840 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [n:127.0.0.1:46719_kg_o%2Fc ] o.a.s.c.Overseer Overseer (id=98511688911486980-127.0.0.1:46719_kg_o%2Fc-n_0000000000) starting
[junit4] 2> 3606875 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [n:127.0.0.1:46719_kg_o%2Fc ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46719_kg_o%2Fc
[junit4] 2> 3606895 INFO (zkCallback-3219-thread-1-processing-n:127.0.0.1:46719_kg_o%2Fc) [n:127.0.0.1:46719_kg_o%2Fc ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3607307 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [n:127.0.0.1:46719_kg_o%2Fc ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2b064754
[junit4] 2> 3607339 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [n:127.0.0.1:46719_kg_o%2Fc ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2b064754
[junit4] 2> 3607339 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [n:127.0.0.1:46719_kg_o%2Fc ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2b064754
[junit4] 2> 3607341 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [n:127.0.0.1:46719_kg_o%2Fc ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ShardSplitTest_A552ECE1FD218366-001/control-001/cores
[junit4] 2> 3607438 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3607438 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37873/solr ready
[junit4] 2> 3607452 INFO (qtp189512190-16374) [n:127.0.0.1:46719_kg_o%2Fc ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:46719_kg_o%252Fc&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 3607454 INFO (OverseerThreadFactory-4569-thread-1-processing-n:127.0.0.1:46719_kg_o%2Fc) [n:127.0.0.1:46719_kg_o%2Fc ] o.a.s.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 3607593 INFO (qtp189512190-16375) [n:127.0.0.1:46719_kg_o%2Fc ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 3607595 INFO (qtp189512190-16375) [n:127.0.0.1:46719_kg_o%2Fc ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 3607710 INFO (zkCallback-3219-thread-1-processing-n:127.0.0.1:46719_kg_o%2Fc) [n:127.0.0.1:46719_kg_o%2Fc ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 3608656 INFO (qtp189512190-16375) [n:127.0.0.1:46719_kg_o%2Fc c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 3608696 INFO (qtp189512190-16375) [n:127.0.0.1:46719_kg_o%2Fc c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
[junit4] 2> 3609121 INFO (qtp189512190-16375) [n:127.0.0.1:46719_kg_o%2Fc c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 3609156 INFO (qtp189512190-16375) [n:127.0.0.1:46719_kg_o%2Fc c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
[junit4] 2> 3609156 INFO (qtp189512190-16375) [n:127.0.0.1:46719_kg_o%2Fc c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2b064754
[junit4] 2> 3609156 INFO (qtp189512190-16375) [n:127.0.0.1:46719_kg_o%2Fc c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 3609156 INFO (qtp189512190-16375) [n:127.0.0.1:46719_kg_o%2Fc c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ShardSplitTest_A552ECE1FD218366-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ShardSplitTest_A552ECE1FD218366-001/control-001/cores/control_collection_shard1_replica_n1/data/]
[junit4] 2> 3609171 INFO (qtp189512190-16375) [n:127.0.0.1:46719_kg_o%2Fc c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=5, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 3609210 WARN (qtp189512190-16375) [n:127.0.0.1:46719_kg_o%2Fc c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 3609325 INFO (qtp189512190-16375) [n:127.0.0.1:46719_kg_o%2Fc c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 3609325 INFO (qtp189512190-16375) [n:127.0.0.1:46719_kg_o%2Fc c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 3609326 INFO (qtp189512190-16375) [n:127.0.0.1:46719_kg_o%2Fc c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 3609326 INFO (qtp189512190-16375) [n:127.0.0.1:46719_kg_o%2Fc c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 3609327 INFO (qtp189512190-16375) [n:127.0.0.1:46719_kg_o%2Fc c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=30, maxMergeAtOnceExplicit=36, maxMergedSegmentMB=13.845703125, floorSegmentMB=0.3818359375, forceMergeDeletesPctAllowed=17.515840046183506, segmentsPerTier=28.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 3609340 INFO (qtp189512190-16375) [n:127.0.0.1:46719_kg_o%2Fc c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1c344f9b[control_collection_shard1_replica_n1] main]
[junit4] 2> 3609341 INFO (qtp189512190-16375) [n:127.0.0.1:46719_kg_o%2Fc c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 3609341 INFO (qtp189512190-16375) [n:127.0.0.1:46719_kg_o%2Fc c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 3609341 INFO (qtp189512190-16375) [n:127.0.0.1:46719_kg_o%2Fc c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 3609342 INFO (qtp189512190-16375) [n:127.0.0.1:46719_kg_o%2Fc c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1576187026839961600
[junit4] 2> 3609357 INFO (searcherExecutor-4572-thread-1-processing-n:127.0.0.1:46719_kg_o%2Fc x:control_collection_shard1_replica_n1 s:shard1 c:control_collection) [n:127.0.0.1:46719_kg_o%2Fc c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@1c344f9b[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 3609359 INFO (qtp189512190-16375) [n:127.0.0.1:46719_kg_o%2Fc c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 3609359 INFO (qtp189512190-16375) [n:127.0.0.1:46719_kg_o%2Fc c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 3609359 INFO (qtp189512190-16375) [n:127.0.0.1:46719_kg_o%2Fc c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:46719/kg_o/c/control_collection_shard1_replica_n1/
[junit4] 2> 3609359 INFO (qtp189512190-16375) [n:127.0.0.1:46719_kg_o%2Fc c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 3609359 INFO (qtp189512190-16375) [n:127.0.0.1:46719_kg_o%2Fc c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:46719/kg_o/c/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 3609359 INFO (qtp189512190-16375) [n:127.0.0.1:46719_kg_o%2Fc c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 3609377 INFO (qtp189512190-16375) [n:127.0.0.1:46719_kg_o%2Fc c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:46719/kg_o/c/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 3609480 INFO (zkCallback-3219-thread-1-processing-n:127.0.0.1:46719_kg_o%2Fc) [n:127.0.0.1:46719_kg_o%2Fc ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 3609531 INFO (qtp189512190-16375) [n:127.0.0.1:46719_kg_o%2Fc c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 3609532 INFO (qtp189512190-16375) [n:127.0.0.1:46719_kg_o%2Fc c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1938
[junit4] 2> 3609556 INFO (qtp189512190-16374) [n:127.0.0.1:46719_kg_o%2Fc ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 3609642 INFO (zkCallback-3219-thread-1-processing-n:127.0.0.1:46719_kg_o%2Fc) [n:127.0.0.1:46719_kg_o%2Fc ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 3610556 INFO (qtp189512190-16374) [n:127.0.0.1:46719_kg_o%2Fc ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:46719_kg_o%252Fc&wt=javabin&version=2} status=0 QTime=3104
[junit4] 2> 3610590 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3610590 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37873/solr ready
[junit4] 2> 3610590 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 3610591 INFO (qtp189512190-16377) [n:127.0.0.1:46719_kg_o%2Fc ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 3610592 INFO (OverseerCollectionConfigSetProcessor-98511688911486980-127.0.0.1:46719_kg_o%2Fc-n_0000000000) [n:127.0.0.1:46719_kg_o%2Fc ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 3610593 INFO (OverseerThreadFactory-4569-thread-2-processing-n:127.0.0.1:46719_kg_o%2Fc) [n:127.0.0.1:46719_kg_o%2Fc ] o.a.s.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 3610593 WARN (OverseerThreadFactory-4569-thread-2-processing-n:127.0.0.1:46719_kg_o%2Fc) [n:127.0.0.1:46719_kg_o%2Fc ] o.a.s.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
[junit4] 2> 3610812 INFO (qtp189512190-16377) [n:127.0.0.1:46719_kg_o%2Fc ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 3610812 INFO (qtp189512190-16377) [n:127.0.0.1:46719_kg_o%2Fc ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=1&wt=javabin&version=2} status=0 QTime=221
[junit4] 2> 3611050 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ShardSplitTest_A552ECE1FD218366-001/shard-1-001 of type NRT
[junit4] 2> 3611051 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 3611076 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@794f9e97{/kg_o/c,null,AVAILABLE}
[junit4] 2> 3611076 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@235cf8a4{HTTP/1.1,[http/1.1]}{127.0.0.1:42149}
[junit4] 2> 3611076 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.e.j.s.Server Started @3623035ms
[junit4] 2> 3611076 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ShardSplitTest_A552ECE1FD218366-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/kg_o/c, hostPort=42149, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ShardSplitTest_A552ECE1FD218366-001/shard-1-001/cores}
[junit4] 2> 3611077 ERROR (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3611077 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
[junit4] 2> 3611077 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3611077 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 3611077 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-19T18:58:07.085Z
[junit4] 2> 3611121 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 3611121 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ShardSplitTest_A552ECE1FD218366-001/shard-1-001/solr.xml
[junit4] 2> 3611125 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@2b064754, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 3611154 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37873/solr
[junit4] 2> 3611197 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [n:127.0.0.1:42149_kg_o%2Fc ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3611198 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [n:127.0.0.1:42149_kg_o%2Fc ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3611200 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [n:127.0.0.1:42149_kg_o%2Fc ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42149_kg_o%2Fc
[junit4] 2> 3611200 INFO (zkCallback-3219-thread-1-processing-n:127.0.0.1:46719_kg_o%2Fc) [n:127.0.0.1:46719_kg_o%2Fc ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 3611201 INFO (zkCallback-3226-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 3611242 INFO (zkCallback-3231-thread-1-processing-n:127.0.0.1:42149_kg_o%2Fc) [n:127.0.0.1:42149_kg_o%2Fc ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 3611362 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [n:127.0.0.1:42149_kg_o%2Fc ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2b064754
[junit4] 2> 3611387 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [n:127.0.0.1:42149_kg_o%2Fc ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2b064754
[junit4] 2> 3611387 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [n:127.0.0.1:42149_kg_o%2Fc ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2b064754
[junit4] 2> 3611417 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [n:127.0.0.1:42149_kg_o%2Fc ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ShardSplitTest_A552ECE1FD218366-001/shard-1-001/cores
[junit4] 2> 3611495 INFO (qtp189512190-16380) [n:127.0.0.1:46719_kg_o%2Fc ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:42149_kg_o%252Fc&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 3611496 INFO (OverseerCollectionConfigSetProcessor-98511688911486980-127.0.0.1:46719_kg_o%2Fc-n_0000000000) [n:127.0.0.1:46719_kg_o%2Fc ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 3611496 INFO (OverseerThreadFactory-4569-thread-3-processing-n:127.0.0.1:46719_kg_o%2Fc) [n:127.0.0.1:46719_kg_o%2Fc ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:42149_kg_o%2Fc for creating new replica
[junit4] 2> 3611506 INFO (qtp40291941-16419) [n:127.0.0.1:42149_kg_o%2Fc ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 3611507 INFO (qtp40291941-16419) [n:127.0.0.1:42149_kg_o%2Fc ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 3612591 INFO (qtp40291941-16419) [n:127.0.0.1:42149_kg_o%2Fc c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 3612680 INFO (qtp40291941-16419) [n:127.0.0.1:42149_kg_o%2Fc c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.s.IndexSchema [collection1_shard2_replica_n41] Schema name=test
[junit4] 2> 3612967 INFO (qtp40291941-16419) [n:127.0.0.1:42149_kg_o%2Fc c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 3613004 INFO (qtp40291941-16419) [n:127.0.0.1:42149_kg_o%2Fc c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n41' using configuration from collection collection1, trusted=true
[junit4] 2> 3613005 INFO (qtp40291941-16419) [n:127.0.0.1:42149_kg_o%2Fc c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_n41' (registry 'solr.core.collection1.shard2.replica_n41') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2b064754
[junit4] 2> 3613005 INFO (qtp40291941-16419) [n:127.0.0.1:42149_kg_o%2Fc c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 3613005 INFO (qtp40291941-16419) [n:127.0.0.1:42149_kg_o%2Fc c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.SolrCore [[collection1_shard2_replica_n41] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ShardSplitTest_A552ECE1FD218366-001/shard-1-001/cores/collection1_shard2_replica_n41], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ShardSplitTest_A552ECE1FD218366-001/shard-1-001/cores/collection1_shard2_replica_n41/data/]
[junit4] 2> 3613020 INFO (qtp40291941-16419) [n:127.0.0.1:42149_kg_o%2Fc c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=5, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 3613041 WARN (qtp40291941-16419) [n:127.0.0.1:42149_kg_o%2Fc c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 3613340 INFO (qtp40291941-16419) [n:127.0.0.1:42149_kg_o%2Fc c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 3613340 INFO (qtp40291941-16419) [n:127.0.0.1:42149_kg_o%2Fc c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 3613341 INFO (qtp40291941-16419) [n:127.0.0.1:42149_kg_o%2Fc c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 3613341 INFO (qtp40291941-16419) [n:127.0.0.1:42149_kg_o%2Fc c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 3613342 INFO (qtp40291941-16419) [n:127.0.0.1:42149_kg_o%2Fc c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=30, maxMergeAtOnceExplicit=36, maxMergedSegmentMB=13.845703125, floorSegmentMB=0.3818359375, forceMergeDeletesPctAllowed=17.515840046183506, segmentsPerTier=28.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 3613355 INFO (qtp40291941-16419) [n:127.0.0.1:42149_kg_o%2Fc c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.s.SolrIndexSearcher Opening [Searcher@23cc22d5[collection1_shard2_replica_n41] main]
[junit4] 2> 3613356 INFO (qtp40291941-16419) [n:127.0.0.1:42149_kg_o%2Fc c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 3613356 INFO (qtp40291941-16419) [n:127.0.0.1:42149_kg_o%2Fc c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 3613357 INFO (qtp40291941-16419) [n:127.0.0.1:42149_kg_o%2Fc c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 3613358 INFO (searcherExecutor-4583-thread-1-processing-n:127.0.0.1:42149_kg_o%2Fc x:collection1_shard2_replica_n41 s:shard2 c:collection1) [n:127.0.0.1:42149_kg_o%2Fc c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.SolrCore [collection1_shard2_replica_n41] Registered new searcher Searcher@23cc22d5[collection1_shard2_replica_n41] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 3613358 INFO (qtp40291941-16419) [n:127.0.0.1:42149_kg_o%2Fc c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1576187031051042816
[junit4] 2> 3613374 INFO (qtp40291941-16419) [n:127.0.0.1:42149_kg_o%2Fc c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 3613374 INFO (qtp40291941-16419) [n:127.0.0.1:42149_kg_o%2Fc c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 3613374 INFO (qtp40291941-16419) [n:127.0.0.1:42149_kg_o%2Fc c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:42149/kg_o/c/collection1_shard2_replica_n41/
[junit4] 2> 3613374 INFO (qtp40291941-16419) [n:127.0.0.1:42149_kg_o%2Fc c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 3613374 INFO (qtp40291941-16419) [n:127.0.0.1:42149_kg_o%2Fc c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.SyncStrategy http://127.0.0.1:42149/kg_o/c/collection1_shard2_replica_n41/ has no replicas
[junit4] 2> 3613374 INFO (qtp40291941-16419) [n:127.0.0.1:42149_kg_o%2Fc c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 3613388 INFO (qtp40291941-16419) [n:127.0.0.1:42149_kg_o%2Fc c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:42149/kg_o/c/collection1_shard2_replica_n41/ shard2
[junit4] 2> 3613552 INFO (qtp40291941-16419) [n:127.0.0.1:42149_kg_o%2Fc c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 3613553 INFO (qtp40291941-16419) [n:127.0.0.1:42149_kg_o%2Fc c:collection1 s:shard2 x:collection1_shard2_replica_n41] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2046
[junit4] 2> 3613571 INFO (qtp189512190-16380) [n:127.0.0.1:46719_kg_o%2Fc ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:42149_kg_o%252Fc&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2} status=0 QTime=2076
[junit4] 2> 3613963 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ShardSplitTest_A552ECE1FD218366-001/shard-2-001 of type NRT
[junit4] 2> 3613963 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 3614038 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7c09dd2a{/kg_o/c,null,AVAILABLE}
[junit4] 2> 3614038 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@b2c61c9{HTTP/1.1,[http/1.1]}{127.0.0.1:46398}
[junit4] 2> 3614038 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.e.j.s.Server Started @3625997ms
[junit4] 2> 3614038 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ShardSplitTest_A552ECE1FD218366-001/tempDir-001/jetty2, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/kg_o/c, hostPort=46398, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ShardSplitTest_A552ECE1FD218366-001/shard-2-001/cores}
[junit4] 2> 3614038 ERROR (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3614054 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[A552ECE1FD218366]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? ver
[...truncated too long message...]
etProxy Closing 0 connections to: http://127.0.0.1:35915/, target: http://127.0.0.1:36419/
[junit4] 2> 6588806 WARN (TEST-LeaderFailoverAfterPartitionTest.test-seed#[A552ECE1FD218366]) [ ] o.a.s.c.SocketProxy Closing 7 connections to: http://127.0.0.1:40499/, target: http://127.0.0.1:41490/
[junit4] 2> 6588807 WARN (TEST-LeaderFailoverAfterPartitionTest.test-seed#[A552ECE1FD218366]) [ ] o.a.s.c.SocketProxy Closing 5 connections to: http://127.0.0.1:37266/, target: http://127.0.0.1:35676/
[junit4] 2> 6588807 WARN (TEST-LeaderFailoverAfterPartitionTest.test-seed#[A552ECE1FD218366]) [ ] o.a.s.c.SocketProxy Closing 1 connections to: http://127.0.0.1:33140/, target: http://127.0.0.1:32811/
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=LeaderFailoverAfterPartitionTest -Dtests.method=test -Dtests.seed=A552ECE1FD218366 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=vi -Dtests.timezone=Asia/Chongqing -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] FAILURE 147s J1 | LeaderFailoverAfterPartitionTest.test <<<
[junit4] > Throwable #1: java.lang.AssertionError: Expected 2 of 3 replicas to be active but only found 1; [core_node3:{"core":"c8n_1x3_lf_shard1_replica_n1","base_url":"http://127.0.0.1:40499","node_name":"127.0.0.1:40499_","state":"active","type":"NRT","leader":"true"}]; clusterState: DocCollection(c8n_1x3_lf//collections/c8n_1x3_lf/state.json/18)={
[junit4] > "pullReplicas":"0",
[junit4] > "replicationFactor":"1",
[junit4] > "shards":{"shard1":{
[junit4] > "range":"80000000-7fffffff",
[junit4] > "state":"active",
[junit4] > "replicas":{
[junit4] > "core_node3":{
[junit4] > "core":"c8n_1x3_lf_shard1_replica_n1",
[junit4] > "base_url":"http://127.0.0.1:40499",
[junit4] > "node_name":"127.0.0.1:40499_",
[junit4] > "state":"active",
[junit4] > "type":"NRT",
[junit4] > "leader":"true"},
[junit4] > "core_node5":{
[junit4] > "core":"c8n_1x3_lf_shard1_replica_n2",
[junit4] > "base_url":"http://127.0.0.1:35915",
[junit4] > "node_name":"127.0.0.1:35915_",
[junit4] > "state":"down",
[junit4] > "type":"NRT"},
[junit4] > "core_node6":{
[junit4] > "state":"down",
[junit4] > "base_url":"http://127.0.0.1:33140",
[junit4] > "core":"c8n_1x3_lf_shard1_replica_n4",
[junit4] > "node_name":"127.0.0.1:33140_",
[junit4] > "type":"NRT"}}}},
[junit4] > "router":{"name":"compositeId"},
[junit4] > "maxShardsPerNode":"1",
[junit4] > "autoAddReplicas":"false",
[junit4] > "nrtReplicas":"3",
[junit4] > "tlogReplicas":"0"}
[junit4] > at __randomizedtesting.SeedInfo.seed([A552ECE1FD218366:2D06D33B53DDEE9E]:0)
[junit4] > at org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.testRf3WithLeaderFailover(LeaderFailoverAfterPartitionTest.java:169)
[junit4] > at org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test(LeaderFailoverAfterPartitionTest.java:56)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderFailoverAfterPartitionTest_A552ECE1FD218366-001
[junit4] 2> Aug 19, 2017 7:47:44 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70), sim=RandomSimilarity(queryNorm=false): {}, locale=vi, timezone=Asia/Chongqing
[junit4] 2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=179711336,total=525860864
[junit4] 2> NOTE: All tests run in this JVM: [TestExactSharedStatsCache, OverseerStatusTest, SystemInfoHandlerTest, TestAuthorizationFramework, ActionThrottleTest, TestStreamBody, TestGraphMLResponseWriter, HdfsUnloadDistributedZkTest, ClassificationUpdateProcessorTest, TestBlobHandler, SolrCLIZkUtilsTest, TestDynamicFieldCollectionResource, LoggingHandlerTest, MoreLikeThisHandlerTest, ZkCLITest, ShardRoutingCustomTest, ConvertedLegacyTest, SolrSlf4jReporterTest, CdcrBootstrapTest, TestSimpleTrackingShardHandler, DistribDocExpirationUpdateProcessorTest, TestUpdate, NoCacheHeaderTest, ImplicitSnitchTest, JsonLoaderTest, TestFieldCacheVsDocValues, DistributedFacetPivotWhiteBoxTest, StatelessScriptUpdateProcessorFactoryTest, SubstringBytesRefFilterTest, SoftAutoCommitTest, TestPrepRecovery, TestFieldSortValues, CurrencyFieldTypeTest, TestHashPartitioner, SchemaVersionSpecificBehaviorTest, TestExclusionRuleCollectionAccess, TestSolrConfigHandler, ScriptEngineTest, CollectionsAPIDistributedZkTest, JSONWriterTest, DistributedVersionInfoTest, TestInPlaceUpdatesDistrib, SuggestComponentContextFilterQueryTest, TestStressVersions, TestComplexPhraseLeadingWildcard, TestCursorMarkWithoutUniqueKey, FileBasedSpellCheckerTest, ZkSolrClientTest, TestManagedStopFilterFactory, DistributedIntervalFacetingTest, DeleteLastCustomShardedReplicaTest, TestRTGBase, AtomicUpdatesTest, SOLR749Test, TestQueryTypes, StatsReloadRaceTest, TestRestManager, TestSchemaVersionResource, WrapperMergePolicyFactoryTest, CacheHeaderTest, CollectionTooManyReplicasTest, DistributedMLTComponentTest, TestCustomStream, DisMaxRequestHandlerTest, ChaosMonkeySafeLeaderTest, TestLegacyFieldReuse, DistributedExpandComponentTest, TestJsonFacetRefinement, TestNoOpRegenerator, TestSolrIndexConfig, TestSQLHandler, SolrCmdDistributorTest, TermVectorComponentTest, TestOrdValues, ReplicationFactorTest, TestRawResponseWriter, BadCopyFieldTest, TestSSLRandomization, HdfsChaosMonkeySafeLeaderTest, GraphQueryTest, TestScoreJoinQPNoScore, VersionInfoTest, TestStressRecovery, TestPayloadScoreQParserPlugin, AnalyticsMergeStrategyTest, TestSolrCloudWithSecureImpersonation, TestCoreDiscovery, BlockDirectoryTest, SolrMetricReporterTest, TestReplicationHandlerBackup, TestTolerantUpdateProcessorRandomCloud, TestManagedSchemaThreadSafety, BitVectorTest, HdfsLockFactoryTest, TestLeaderElectionZkExpiry, TestUnifiedSolrHighlighter, SSLMigrationTest, CdcrRequestHandlerTest, HdfsSyncSliceTest, BadComponentTest, SharedFSAutoReplicaFailoverTest, DocValuesTest, UpdateRequestProcessorFactoryTest, ExitableDirectoryReaderTest, DocValuesMultiTest, DistributedSuggestComponentTest, TestClusterStateMutator, DeleteShardTest, TestConfigSetsAPIExclusivity, DistanceFunctionTest, BlobRepositoryCloudTest, TestPostingsSolrHighlighter, ResourceLoaderTest, TestDFISimilarityFactory, TestFuzzyAnalyzedSuggestions, AnalysisAfterCoreReloadTest, TestTrieFacet, BasicDistributedZk2Test, UnloadDistributedZkTest, OpenCloseCoreStressTest, ClusterStateUpdateTest, TestRandomFaceting, DistributedSpellCheckComponentTest, TermVectorComponentDistributedTest, TestReload, TestCoreContainer, SimpleFacetsTest, TestSolr4Spatial, SolrCoreTest, SpellCheckComponentTest, TestGroupingSearch, DirectUpdateHandlerTest, ShowFileRequestHandlerTest, SimplePostToolTest, SpellCheckCollatorTest, SuggesterFSTTest, TestFoldingMultitermQuery, SuggesterTSTTest, TestStressLucene, TestTrie, SuggesterWFSTTest, WordBreakSolrSpellCheckerTest, TestAtomicUpdateErrorCases, QueryEqualityTest, TestSolrDeletionPolicy1, IndexBasedSpellCheckerTest, LukeRequestHandlerTest, TermsComponentTest, XmlUpdateRequestHandlerTest, DocumentBuilderTest, TestIndexingPerformance, RegexBoostProcessorTest, SolrPluginUtilsTest, TestJmxIntegration, QueryParsingTest, UniqFieldsUpdateProcessorFactoryTest, TestPHPSerializedResponseWriter, CSVRequestHandlerTest, SearchHandlerTest, HighlighterConfigTest, UpdateParamsTest, AlternateDirectoryTest, TestQuerySenderNoQuery, CopyFieldTest, ResponseLogComponentTest, MultiTermTest, SampleTest, TestDocSet, TestBinaryField, TestSearchPerf, MinimalSchemaTest, TestElisionMultitermQuery, TestConfig, OutputWriterTest, ExternalFileFieldSortTest, TestSolrCoreProperties, DirectSolrConnectionTest, TestLuceneMatchVersion, TestPhraseSuggestions, TestCharFilters, TestCodecSupport, TestXIncludeConfig, EchoParamsTest, TestSweetSpotSimilarityFactory, TestPerFieldSimilarity, TestLMDirichletSimilarityFactory, TestBM25SimilarityFactory, TestFastWriter, OpenExchangeRatesOrgProviderTest, PluginInfoTest, TestFastLRUCache, TestSystemIdResolver, PrimUtilsTest, TestSuggestSpellingConverter, DateFieldTest, SpellingQueryConverterTest, DOMUtilTest, ClusterStateTest, TestSolrJ, TestLRUCache, ZkNodePropsTest, SliceStateTest, UUIDFieldTest, FileUtilsTest, CircularListTest, CursorPagingTest, SolrTestCaseJ4Test, TestEmbeddedSolrServerConstructors, TestEmbeddedSolrServerSchemaAPI, TestJettySolrRunner, AssignBackwardCompatibilityTest, AssignTest, CdcrReplicationDistributedZkTest, CdcrVersionReplicationTest, CleanupOldIndexTest, CloudExitableDirectoryReaderTest, CollectionReloadTest, CollectionsAPISolrJTest, ConnectionManagerTest, DeleteReplicaTest, ForceLeaderTest, LeaderElectionContextKeyTest, LeaderFailoverAfterPartitionTest]
[junit4] Completed [508/729 (4!)] on J1 in 147.88s, 1 test, 1 failure <<< FAILURES!
[...truncated 44110 lines...]
[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 30 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/30/
3 tests failed.
FAILED: org.apache.solr.cloud.HttpPartitionTest.test
Error Message:
Doc with id=1 not found in http://127.0.0.1:49734/zki/h/collMinRf_1x3 due to: Path not found: /id; rsp={doc=null}
Stack Trace:
java.lang.AssertionError: Doc with id=1 not found in http://127.0.0.1:49734/zki/h/collMinRf_1x3 due to: Path not found: /id; rsp={doc=null}
at __randomizedtesting.SeedInfo.seed([BB1D69E7C5127864:3349563D6BEE159C]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.HttpPartitionTest.assertDocExists(HttpPartitionTest.java:603)
at org.apache.solr.cloud.HttpPartitionTest.assertDocsExistInAllReplicas(HttpPartitionTest.java:558)
at org.apache.solr.cloud.HttpPartitionTest.testMinRf(HttpPartitionTest.java:249)
at org.apache.solr.cloud.HttpPartitionTest.test(HttpPartitionTest.java:127)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader
Error Message:
Doc with id=1 not found in http://127.0.0.1:60844/forceleader_test_collection due to: Path not found: /id; rsp={doc=null}
Stack Trace:
java.lang.AssertionError: Doc with id=1 not found in http://127.0.0.1:60844/forceleader_test_collection due to: Path not found: /id; rsp={doc=null}
at __randomizedtesting.SeedInfo.seed([BB1D69E7C5127864:5D8A5D27FC908105]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.HttpPartitionTest.assertDocExists(HttpPartitionTest.java:603)
at org.apache.solr.cloud.HttpPartitionTest.assertDocsExistInAllReplicas(HttpPartitionTest.java:556)
at org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader(ForceLeaderTest.java:142)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test
Error Message:
expected:<2> but was:<3>
Stack Trace:
java.lang.AssertionError: expected:<2> but was:<3>
at __randomizedtesting.SeedInfo.seed([BB1D69E7C5127864:3349563D6BEE159C]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.assertSliceAndReplicaCount(SharedFSAutoReplicaFailoverTest.java:403)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.testBasics(SharedFSAutoReplicaFailoverTest.java:248)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test(SharedFSAutoReplicaFailoverTest.java:136)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 11465 lines...]
[junit4] Suite: org.apache.solr.cloud.HttpPartitionTest
[junit4] 2> 437323 INFO (SUITE-HttpPartitionTest-seed#[BB1D69E7C5127864]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_BB1D69E7C5127864-001/init-core-data-001
[junit4] 2> 437324 WARN (SUITE-HttpPartitionTest-seed#[BB1D69E7C5127864]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
[junit4] 2> 437324 INFO (SUITE-HttpPartitionTest-seed#[BB1D69E7C5127864]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 437325 INFO (SUITE-HttpPartitionTest-seed#[BB1D69E7C5127864]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
[junit4] 2> 437325 INFO (SUITE-HttpPartitionTest-seed#[BB1D69E7C5127864]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /zki/h
[junit4] 2> 437327 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 437327 INFO (Thread-413) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 437327 INFO (Thread-413) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 437329 ERROR (Thread-413) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 437427 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.ZkTestServer start zk server on port:45679
[junit4] 2> 437447 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 437449 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 437450 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 437451 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 437452 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 437453 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 437454 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 437459 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 437460 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 437461 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 437463 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 437464 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15de6dddce80001, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 437468 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use TLOG replicas unless explicitly asked otherwise
[junit4] 2> 437552 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 437553 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@66581355{/zki/h,null,AVAILABLE}
[junit4] 2> 437555 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@5e1147b9{HTTP/1.1,[http/1.1]}{127.0.0.1:37701}
[junit4] 2> 437555 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.e.j.s.Server Started @440835ms
[junit4] 2> 437555 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_BB1D69E7C5127864-001/tempDir-001/control/data, replicaType=NRT, hostContext=/zki/h, hostPort=52717, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_BB1D69E7C5127864-001/control-001/cores}
[junit4] 2> 437555 ERROR (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 437555 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
[junit4] 2> 437555 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 437556 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 437556 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-15T17:08:06.220Z
[junit4] 2> 437558 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 437558 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15de6dddce80002, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 437558 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_BB1D69E7C5127864-001/control-001/solr.xml
[junit4] 2> 437563 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@77971aa7, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 437565 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45679/solr
[junit4] 2> 437623 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [n:127.0.0.1:52717_zki%2Fh ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 437623 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [n:127.0.0.1:52717_zki%2Fh ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:52717_zki%2Fh
[junit4] 2> 437624 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [n:127.0.0.1:52717_zki%2Fh ] o.a.s.c.Overseer Overseer (id=98488607440371716-127.0.0.1:52717_zki%2Fh-n_0000000000) starting
[junit4] 2> 437630 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [n:127.0.0.1:52717_zki%2Fh ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52717_zki%2Fh
[junit4] 2> 437631 INFO (zkCallback-331-thread-1-processing-n:127.0.0.1:52717_zki%2Fh) [n:127.0.0.1:52717_zki%2Fh ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 437675 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [n:127.0.0.1:52717_zki%2Fh ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@77971aa7
[junit4] 2> 437683 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [n:127.0.0.1:52717_zki%2Fh ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@77971aa7
[junit4] 2> 437683 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [n:127.0.0.1:52717_zki%2Fh ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@77971aa7
[junit4] 2> 437684 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [n:127.0.0.1:52717_zki%2Fh ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_BB1D69E7C5127864-001/control-001/cores
[junit4] 2> 437709 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 437709 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:45679/solr ready
[junit4] 2> 437712 INFO (SocketProxy-Acceptor-52717) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=40755,localport=52717], receiveBufferSize:531000
[junit4] 2> 437715 INFO (SocketProxy-Acceptor-52717) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=37701,localport=36677], receiveBufferSize=530904
[junit4] 2> 437716 INFO (qtp592632440-3200) [n:127.0.0.1:52717_zki%2Fh ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:52717_zki%252Fh&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 437718 INFO (OverseerThreadFactory-1087-thread-1-processing-n:127.0.0.1:52717_zki%2Fh) [n:127.0.0.1:52717_zki%2Fh ] o.a.s.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 437822 INFO (SocketProxy-Acceptor-52717) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=40757,localport=52717], receiveBufferSize:531000
[junit4] 2> 437824 INFO (SocketProxy-Acceptor-52717) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=37701,localport=36679], receiveBufferSize=530904
[junit4] 2> 437827 INFO (qtp592632440-3202) [n:127.0.0.1:52717_zki%2Fh ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 437827 INFO (qtp592632440-3202) [n:127.0.0.1:52717_zki%2Fh ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 437931 INFO (zkCallback-331-thread-1-processing-n:127.0.0.1:52717_zki%2Fh) [n:127.0.0.1:52717_zki%2Fh ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 438856 INFO (qtp592632440-3202) [n:127.0.0.1:52717_zki%2Fh c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 438891 INFO (qtp592632440-3202) [n:127.0.0.1:52717_zki%2Fh c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
[junit4] 2> 438990 INFO (qtp592632440-3202) [n:127.0.0.1:52717_zki%2Fh c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 439002 INFO (qtp592632440-3202) [n:127.0.0.1:52717_zki%2Fh c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
[junit4] 2> 439002 INFO (qtp592632440-3202) [n:127.0.0.1:52717_zki%2Fh c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@77971aa7
[junit4] 2> 439002 INFO (qtp592632440-3202) [n:127.0.0.1:52717_zki%2Fh c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 439002 INFO (qtp592632440-3202) [n:127.0.0.1:52717_zki%2Fh c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_BB1D69E7C5127864-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_BB1D69E7C5127864-001/control-001/cores/control_collection_shard1_replica_n1/data/]
[junit4] 2> 439004 INFO (qtp592632440-3202) [n:127.0.0.1:52717_zki%2Fh c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@67e7c5d0
[junit4] 2> 439007 WARN (qtp592632440-3202) [n:127.0.0.1:52717_zki%2Fh c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 439038 INFO (qtp592632440-3202) [n:127.0.0.1:52717_zki%2Fh c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 439038 INFO (qtp592632440-3202) [n:127.0.0.1:52717_zki%2Fh c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 439038 INFO (qtp592632440-3202) [n:127.0.0.1:52717_zki%2Fh c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 439039 INFO (qtp592632440-3202) [n:127.0.0.1:52717_zki%2Fh c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 439040 INFO (qtp592632440-3202) [n:127.0.0.1:52717_zki%2Fh c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=14, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 439040 INFO (qtp592632440-3202) [n:127.0.0.1:52717_zki%2Fh c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3cf6b672[control_collection_shard1_replica_n1] main]
[junit4] 2> 439041 INFO (qtp592632440-3202) [n:127.0.0.1:52717_zki%2Fh c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 439041 INFO (qtp592632440-3202) [n:127.0.0.1:52717_zki%2Fh c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 439042 INFO (qtp592632440-3202) [n:127.0.0.1:52717_zki%2Fh c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 439043 INFO (searcherExecutor-1090-thread-1-processing-n:127.0.0.1:52717_zki%2Fh x:control_collection_shard1_replica_n1 s:shard1 c:control_collection) [n:127.0.0.1:52717_zki%2Fh c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@3cf6b672[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 439044 INFO (qtp592632440-3202) [n:127.0.0.1:52717_zki%2Fh c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575817720845303808
[junit4] 2> 439050 INFO (qtp592632440-3202) [n:127.0.0.1:52717_zki%2Fh c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 439050 INFO (qtp592632440-3202) [n:127.0.0.1:52717_zki%2Fh c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 439050 INFO (qtp592632440-3202) [n:127.0.0.1:52717_zki%2Fh c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:52717/zki/h/control_collection_shard1_replica_n1/
[junit4] 2> 439050 INFO (qtp592632440-3202) [n:127.0.0.1:52717_zki%2Fh c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 439050 INFO (qtp592632440-3202) [n:127.0.0.1:52717_zki%2Fh c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:52717/zki/h/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 439050 INFO (qtp592632440-3202) [n:127.0.0.1:52717_zki%2Fh c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 439052 INFO (qtp592632440-3202) [n:127.0.0.1:52717_zki%2Fh c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:52717/zki/h/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 439154 INFO (zkCallback-331-thread-1-processing-n:127.0.0.1:52717_zki%2Fh) [n:127.0.0.1:52717_zki%2Fh ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 439203 INFO (qtp592632440-3202) [n:127.0.0.1:52717_zki%2Fh c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 439204 INFO (qtp592632440-3202) [n:127.0.0.1:52717_zki%2Fh c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1377
[junit4] 2> 439207 INFO (qtp592632440-3200) [n:127.0.0.1:52717_zki%2Fh ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 439306 INFO (zkCallback-331-thread-1-processing-n:127.0.0.1:52717_zki%2Fh) [n:127.0.0.1:52717_zki%2Fh ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 439720 INFO (OverseerCollectionConfigSetProcessor-98488607440371716-127.0.0.1:52717_zki%2Fh-n_0000000000) [n:127.0.0.1:52717_zki%2Fh ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 440207 INFO (qtp592632440-3200) [n:127.0.0.1:52717_zki%2Fh ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:52717_zki%252Fh&wt=javabin&version=2} status=0 QTime=2491
[junit4] 2> 440213 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 440214 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:45679/solr ready
[junit4] 2> 440214 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 440215 INFO (SocketProxy-Acceptor-52717) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=40765,localport=52717], receiveBufferSize:531000
[junit4] 2> 440215 INFO (SocketProxy-Acceptor-52717) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=37701,localport=36687], receiveBufferSize=530904
[junit4] 2> 440216 INFO (qtp592632440-3198) [n:127.0.0.1:52717_zki%2Fh ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 440219 INFO (OverseerThreadFactory-1087-thread-2-processing-n:127.0.0.1:52717_zki%2Fh) [n:127.0.0.1:52717_zki%2Fh ] o.a.s.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 440220 WARN (OverseerThreadFactory-1087-thread-2-processing-n:127.0.0.1:52717_zki%2Fh) [n:127.0.0.1:52717_zki%2Fh ] o.a.s.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
[junit4] 2> 440424 INFO (qtp592632440-3198) [n:127.0.0.1:52717_zki%2Fh ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 440424 INFO (qtp592632440-3198) [n:127.0.0.1:52717_zki%2Fh ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=2&wt=javabin&version=2} status=0 QTime=208
[junit4] 2> 440506 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_BB1D69E7C5127864-001/shard-1-001 of type TLOG
[junit4] 2> 440506 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 440508 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2afeb41b{/zki/h,null,AVAILABLE}
[junit4] 2> 440508 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7eb37e46{HTTP/1.1,[http/1.1]}{127.0.0.1:54299}
[junit4] 2> 440508 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.e.j.s.Server Started @443788ms
[junit4] 2> 440508 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_BB1D69E7C5127864-001/tempDir-001/jetty1, replicaType=TLOG, solrconfig=solrconfig.xml, hostContext=/zki/h, hostPort=52426, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_BB1D69E7C5127864-001/shard-1-001/cores}
[junit4] 2> 440509 ERROR (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 440509 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
[junit4] 2> 440509 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 440509 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 440509 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-15T17:08:09.173Z
[junit4] 2> 440511 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 440511 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_BB1D69E7C5127864-001/shard-1-001/solr.xml
[junit4] 2> 440515 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@77971aa7, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 440517 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45679/solr
[junit4] 2> 440524 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [n:127.0.0.1:52426_zki%2Fh ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 440525 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [n:127.0.0.1:52426_zki%2Fh ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 440527 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [n:127.0.0.1:52426_zki%2Fh ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52426_zki%2Fh
[junit4] 2> 440528 INFO (zkCallback-331-thread-1-processing-n:127.0.0.1:52717_zki%2Fh) [n:127.0.0.1:52717_zki%2Fh ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 440528 INFO (zkCallback-338-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 440535 INFO (zkCallback-343-thread-1-processing-n:127.0.0.1:52426_zki%2Fh) [n:127.0.0.1:52426_zki%2Fh ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 440576 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [n:127.0.0.1:52426_zki%2Fh ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@77971aa7
[junit4] 2> 440584 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [n:127.0.0.1:52426_zki%2Fh ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@77971aa7
[junit4] 2> 440584 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [n:127.0.0.1:52426_zki%2Fh ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@77971aa7
[junit4] 2> 440585 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [n:127.0.0.1:52426_zki%2Fh ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_BB1D69E7C5127864-001/shard-1-001/cores
[junit4] 2> 440604 INFO (SocketProxy-Acceptor-52426) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=42496,localport=52426], receiveBufferSize:531000
[junit4] 2> 440605 INFO (SocketProxy-Acceptor-52426) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=54299,localport=49694], receiveBufferSize=530904
[junit4] 2> 440607 INFO (qtp365366108-3248) [n:127.0.0.1:52426_zki%2Fh ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:52426_zki%252Fh&action=ADDREPLICA&collection=collection1&shard=shard2&type=TLOG&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 440608 INFO (OverseerCollectionConfigSetProcessor-98488607440371716-127.0.0.1:52717_zki%2Fh-n_0000000000) [n:127.0.0.1:52717_zki%2Fh ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 440609 INFO (OverseerThreadFactory-1087-thread-3-processing-n:127.0.0.1:52717_zki%2Fh) [n:127.0.0.1:52717_zki%2Fh ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:52426_zki%2Fh for creating new replica
[junit4] 2> 440610 INFO (SocketProxy-Acceptor-52426) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=42498,localport=52426], receiveBufferSize:531000
[junit4] 2> 440611 INFO (SocketProxy-Acceptor-52426) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=54299,localport=49696], receiveBufferSize=530904
[junit4] 2> 440611 INFO (qtp365366108-3246) [n:127.0.0.1:52426_zki%2Fh ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_t41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=TLOG
[junit4] 2> 440612 INFO (qtp365366108-3246) [n:127.0.0.1:52426_zki%2Fh ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 440716 INFO (zkCallback-343-thread-1-processing-n:127.0.0.1:52426_zki%2Fh) [n:127.0.0.1:52426_zki%2Fh ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 441624 INFO (qtp365366108-3246) [n:127.0.0.1:52426_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 441647 INFO (qtp365366108-3246) [n:127.0.0.1:52426_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.s.IndexSchema [collection1_shard2_replica_t41] Schema name=test
[junit4] 2> 441765 INFO (qtp365366108-3246) [n:127.0.0.1:52426_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 441778 INFO (qtp365366108-3246) [n:127.0.0.1:52426_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_t41' using configuration from collection collection1, trusted=true
[junit4] 2> 441778 INFO (qtp365366108-3246) [n:127.0.0.1:52426_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_t41' (registry 'solr.core.collection1.shard2.replica_t41') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@77971aa7
[junit4] 2> 441778 INFO (qtp365366108-3246) [n:127.0.0.1:52426_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 441778 INFO (qtp365366108-3246) [n:127.0.0.1:52426_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.SolrCore [[collection1_shard2_replica_t41] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_BB1D69E7C5127864-001/shard-1-001/cores/collection1_shard2_replica_t41], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_BB1D69E7C5127864-001/shard-1-001/cores/collection1_shard2_replica_t41/data/]
[junit4] 2> 441781 INFO (qtp365366108-3246) [n:127.0.0.1:52426_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@1c300b2e
[junit4] 2> 441782 WARN (qtp365366108-3246) [n:127.0.0.1:52426_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 441840 INFO (qtp365366108-3246) [n:127.0.0.1:52426_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 441840 INFO (qtp365366108-3246) [n:127.0.0.1:52426_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 441841 INFO (qtp365366108-3246) [n:127.0.0.1:52426_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 441841 INFO (qtp365366108-3246) [n:127.0.0.1:52426_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 441842 INFO (qtp365366108-3246) [n:127.0.0.1:52426_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=14, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 441842 INFO (qtp365366108-3246) [n:127.0.0.1:52426_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.s.SolrIndexSearcher Opening [Searcher@5c73c9a[collection1_shard2_replica_t41] main]
[junit4] 2> 441843 INFO (qtp365366108-3246) [n:127.0.0.1:52426_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 441843 INFO (qtp365366108-3246) [n:127.0.0.1:52426_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 441844 INFO (qtp365366108-3246) [n:127.0.0.1:52426_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 441844 INFO (qtp365366108-3246) [n:127.0.0.1:52426_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575817723781316608
[junit4] 2> 441848 INFO (searcherExecutor-1101-thread-1-processing-n:127.0.0.1:52426_zki%2Fh x:collection1_shard2_replica_t41 s:shard2 c:collection1) [n:127.0.0.1:52426_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.SolrCore [collection1_shard2_replica_t41] Registered new searcher Searcher@5c73c9a[collection1_shard2_replica_t41] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 441851 INFO (qtp365366108-3246) [n:127.0.0.1:52426_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 441851 INFO (qtp365366108-3246) [n:127.0.0.1:52426_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 441851 INFO (qtp365366108-3246) [n:127.0.0.1:52426_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:52426/zki/h/collection1_shard2_replica_t41/
[junit4] 2> 441851 INFO (qtp365366108-3246) [n:127.0.0.1:52426_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 441851 INFO (qtp365366108-3246) [n:127.0.0.1:52426_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.SyncStrategy http://127.0.0.1:52426/zki/h/collection1_shard2_replica_t41/ has no replicas
[junit4] 2> 441851 INFO (qtp365366108-3246) [n:127.0.0.1:52426_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 441852 INFO (qtp365366108-3246) [n:127.0.0.1:52426_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.ZkController collection1_shard2_replica_t41 stopping background replication from leader
[junit4] 2> 441860 INFO (qtp365366108-3246) [n:127.0.0.1:52426_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:52426/zki/h/collection1_shard2_replica_t41/ shard2
[junit4] 2> 441957 INFO (zkCallback-343-thread-1-processing-n:127.0.0.1:52426_zki%2Fh) [n:127.0.0.1:52426_zki%2Fh ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 441963 INFO (qtp365366108-3246) [n:127.0.0.1:52426_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 441964 INFO (qtp365366108-3246) [n:127.0.0.1:52426_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_t41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=1352
[junit4] 2> 441968 INFO (qtp365366108-3248) [n:127.0.0.1:52426_zki%2Fh ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:52426_zki%252Fh&action=ADDREPLICA&collection=collection1&shard=shard2&type=TLOG&wt=javabin&version=2} status=0 QTime=1361
[junit4] 2> 441970 INFO (OverseerCollectionConfigSetProcessor-98488607440371716-127.0.0.1:52717_zki%2Fh-n_0000000000) [n:127.0.0.1:52717_zki%2Fh ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 442071 INFO (zkCallback-343-thread-1-processing-n:127.0.0.1:52426_zki%2Fh) [n:127.0.0.1:52426_zki%2Fh ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 442112 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_BB1D69E7C5127864-001/shard-2-001 of type TLOG
[junit4] 2> 442114 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 442115 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3137df01{/zki/h,null,AVAILABLE}
[junit4] 2> 442115 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@4392886e{HTTP/1.1,[http/1.1]}{127.0.0.1:51516}
[junit4] 2> 442115 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.e.j.s.Server Started @445396ms
[junit4] 2> 442116 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_BB1D69E7C5127864-001/tempDir-001/jetty2, replicaType=TLOG, solrconfig=solrconfig.xml, hostContext=/zki/h, hostPort=49734, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_BB1D69E7C5127864-001/shard-2-001/cores}
[junit4] 2> 442116 ERROR (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 442134 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
[junit4] 2> 442134 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 442134 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 442134 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-15T17:08:10.798Z
[junit4] 2> 442151 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 442151 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_BB1D69E7C5127864-001/shard-2-001/solr.xml
[junit4] 2> 442155 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@77971aa7, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 442157 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45679/solr
[junit4] 2> 442163 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [n:127.0.0.1:49734_zki%2Fh ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 442164 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [n:127.0.0.1:49734_zki%2Fh ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 442166 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [n:127.0.0.1:49734_zki%2Fh ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49734_zki%2Fh
[junit4] 2> 442166 INFO (zkCallback-338-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 442179 INFO (zkCallback-343-thread-1-processing-n:127.0.0.1:52426_zki%2Fh) [n:127.0.0.1:52426_zki%2Fh ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 442179 INFO (zkCallback-331-thread-1-processing-n:127.0.0.1:52717_zki%2Fh) [n:127.0.0.1:52717_zki%2Fh ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 442179 INFO (zkCallback-349-thread-2-processing-n:127.0.0.1:49734_zki%2Fh) [n:127.0.0.1:49734_zki%2Fh ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 442275 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [n:127.0.0.1:49734_zki%2Fh ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@77971aa7
[junit4] 2> 442286 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [n:127.0.0.1:49734_zki%2Fh ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@77971aa7
[junit4] 2> 442286 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [n:127.0.0.1:49734_zki%2Fh ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@77971aa7
[junit4] 2> 442288 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [n:127.0.0.1:49734_zki%2Fh ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_BB1D69E7C5127864-001/shard-2-001/cores
[junit4] 2> 442321 INFO (SocketProxy-Acceptor-49734) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=33655,localport=49734], receiveBufferSize:531000
[junit4] 2> 442339 INFO (SocketProxy-Acceptor-49734) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=51516,localport=47228], receiveBufferSize=530904
[junit4] 2> 442341 INFO (qtp2119428466-3282) [n:127.0.0.1:49734_zki%2Fh ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:49734_zki%252Fh&action=ADDREPLICA&collection=collection1&shard=shard1&type=TLOG&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 442351 INFO (OverseerThreadFactory-1087-thread-4-processing-n:127.0.0.1:52717_zki%2Fh) [n:127.0.0.1:52717_zki%2Fh ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:49734_zki%2Fh for creating new replica
[junit4] 2> 442353 INFO (SocketProxy-Acceptor-49734) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=33663,localport=49734], receiveBufferSize:531000
[junit4] 2> 442373 INFO (SocketProxy-Acceptor-49734) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=51516,localport=47236], receiveBufferSize=530904
[junit4] 2> 442387 INFO (qtp2119428466-3280) [n:127.0.0.1:49734_zki%2Fh ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_t43&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=TLOG
[junit4] 2> 442388 INFO (qtp2119428466-3280) [n:127.0.0.1:49734_zki%2Fh ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 442491 INFO (zkCallback-349-thread-2-processing-n:127.0.0.1:49734_zki%2Fh) [n:127.0.0.1:49734_zki%2Fh ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 442491 INFO (zkCallback-343-thread-1-processing-n:127.0.0.1:52426_zki%2Fh) [n:127.0.0.1:52426_zki%2Fh ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 443413 INFO (qtp2119428466-3280) [n:127.0.0.1:49734_zki%2Fh c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 443475 INFO (qtp2119428466-3280) [n:127.0.0.1:49734_zki%2Fh c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.s.IndexSchema [collection1_shard1_replica_t43] Schema name=test
[junit4] 2> 443903 INFO (qtp2119428466-3280) [n:127.0.0.1:49734_zki%2Fh c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 443974 INFO (qtp2119428466-3280) [n:127.0.0.1:49734_zki%2Fh c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_t43' using configuration from collection collection1, trusted=true
[junit4] 2> 443977 INFO (qtp2119428466-3280) [n:127.0.0.1:49734_zki%2Fh c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_t43' (registry 'solr.core.collection1.shard1.replica_t43') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@77971aa7
[junit4] 2> 443977 INFO (qtp2119428466-3280) [n:127.0.0.1:49734_zki%2Fh c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 443978 INFO (qtp2119428466-3280) [n:127.0.0.1:49734_zki%2Fh c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.SolrCore [[collection1_shard1_replica_t43] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_BB1D69E7C5127864-001/shard-2-001/cores/collection1_shard1_replica_t43], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_BB1D69E7C5127864-001/shard-2-001/cores/collection1_shard1_replica_t43/data/]
[junit4] 2> 443980 INFO (qtp2119428466-3280) [n:127.0.0.1:49734_zki%2Fh c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@73d5047b
[junit4] 2> 444043 WARN (qtp2119428466-3280) [n:127.0.0.1:49734_zki%2Fh c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 444141 INFO (qtp2119428466-3280) [n:127.0.0.1:49734_zki%2Fh c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 444141 INFO (qtp2119428466-3280) [n:127.0.0.1:49734_zki%2Fh c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 444142 INFO (qtp2119428466-3280) [n:127.0.0.1:49734_zki%2Fh c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 444142 INFO (qtp2119428466-3280) [n:127.0.0.1:49734_zki%2Fh c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 444160 INFO (qtp2119428466-3280) [n:127.0.0.1:49734_zki%2Fh c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=14, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 444162 INFO (qtp2119428466-3280) [n:127.0.0.1:49734_zki%2Fh c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.s.SolrIndexSearcher Opening [Searcher@103922f5[collection1_shard1_replica_t43] main]
[junit4] 2> 444164 INFO (qtp2119428466-3280) [n:127.0.0.1:49734_zki%2Fh c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 444164 INFO (qtp2119428466-3280) [n:127.0.0.1:49734_zki%2Fh c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 444165 INFO (qtp2119428466-3280) [n:127.0.0.1:49734_zki%2Fh c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 444165 INFO (qtp2119428466-3280) [n:127.0.0.1:49734_zki%2Fh c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575817726215061504
[junit4] 2> 444191 INFO (qtp2119428466-3280) [n:127.0.0.1:49734_zki%2Fh c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 444191 INFO (qtp2119428466-3280) [n:127.0.0.1:49734_zki%2Fh c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 444191 INFO (qtp2119428466-3280) [n:127.0.0.1:49734_zki%2Fh c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:49734/zki/h/collection1_shard1_replica_t43/
[junit4] 2> 444191 INFO (qtp2119428466-3280) [n:127.0.0.1:49734_zki%2Fh c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 444191 INFO (qtp2119428466-3280) [n:127.0.0.1:49734_zki%2Fh c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.SyncStrategy http://127.0.0.1:49734/zki/h/collection1_shard1_replica_t43/ has no replicas
[junit4] 2> 444191 INFO (qtp2119428466-3280) [n:127.0.0.1:49734_zki%2Fh c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 444192 INFO (qtp2119428466-3280) [n:127.0.0.1:49734_zki%2Fh c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.ZkController collection1_shard1_replica_t43 stopping background replication from leader
[junit4] 2> 444195 INFO (qtp2119428466-3280) [n:127.0.0.1:49734_zki%2Fh c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:49734/zki/h/collection1_shard1_replica_t43/ shard1
[junit4] 2> 444197 INFO (searcherExecutor-1112-thread-1-processing-n:127.0.0.1:49734_zki%2Fh x:collection1_shard1_replica_t43 s:shard1 c:collection1) [n:127.0.0.1:49734_zki%2Fh c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.SolrCore [collection1_shard1_replica_t43] Registered new searcher Searcher@103922f5[collection1_shard1_replica_t43] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 444295 INFO (zkCallback-349-thread-2-processing-n:127.0.0.1:49734_zki%2Fh) [n:127.0.0.1:49734_zki%2Fh ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 444295 INFO (zkCallback-343-thread-1-processing-n:127.0.0.1:52426_zki%2Fh) [n:127.0.0.1:52426_zki%2Fh ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 444345 INFO (qtp2119428466-3280) [n:127.0.0.1:49734_zki%2Fh c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 444351 INFO (qtp2119428466-3280) [n:127.0.0.1:49734_zki%2Fh c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_t43&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=1963
[junit4] 2> 444366 INFO (qtp2119428466-3282) [n:127.0.0.1:49734_zki%2Fh ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:49734_zki%252Fh&action=ADDREPLICA&collection=collection1&shard=shard1&type=TLOG&wt=javabin&version=2} status=0 QTime=2025
[junit4] 2> 444450 INFO (zkCallback-349-thread-2-processing-n:127.0.0.1:49734_zki%2Fh) [n:127.0.0.1:49734_zki%2Fh ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 444450 INFO (zkCallback-343-thread-1-processing-n:127.0.0.1:52426_zki%2Fh) [n:127.0.0.1:52426_zki%2Fh ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 444595 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_BB1D69E7C5127864-001/shard-3-001 of type TLOG
[junit4] 2> 444595 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 444640 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@21a955e7{/zki/h,null,AVAILABLE}
[junit4] 2> 444640 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1f0364a2{HTTP/1.1,[http/1.1]}{127.0.0.1:44225}
[junit4] 2> 444640 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.e.j.s.Server Started @447920ms
[junit4] 2> 444640 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_BB1D69E7C5127864-001/tempDir-001/jetty3, replicaType=TLOG, solrconfig=solrconfig.xml, hostContext=/zki/h, hostPort=38021, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_BB1D69E7C5127864-001/shard-3-001/cores}
[junit4] 2> 444640 ERROR (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 444656 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
[junit4] 2> 444656 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 444656 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 444656 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-15T17:08:13.320Z
[junit4] 2> 444696 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 444696 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_BB1D69E7C5127864-001/shard-3-001/solr.xml
[junit4] 2> 444701 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@77971aa7, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 444717 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45679/solr
[junit4] 2> 444833 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [n:127.0.0.1:38021_zki%2Fh ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 444836 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [n:127.0.0.1:38021_zki%2Fh ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 444838 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [n:127.0.0.1:38021_zki%2Fh ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38021_zki%2Fh
[junit4] 2> 444839 INFO (zkCallback-343-thread-1-processing-n:127.0.0.1:52426_zki%2Fh) [n:127.0.0.1:52426_zki%2Fh ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 444839 INFO (zkCallback-349-thread-2-processing-n:127.0.0.1:49734_zki%2Fh) [n:127.0.0.1:49734_zki%2Fh ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 444840 INFO (zkCallback-331-thread-1-processing-n:127.0.0.1:52717_zki%2Fh) [n:127.0.0.1:52717_zki%2Fh ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 444840 INFO (zkCallback-338-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 444847 INFO (zkCallback-355-thread-1-processing-n:127.0.0.1:38021_zki%2Fh) [n:127.0.0.1:38021_zki%2Fh ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 445551 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [n:127.0.0.1:38021_zki%2Fh ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@77971aa7
[junit4] 2> 445570 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [n:127.0.0.1:38021_zki%2Fh ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@77971aa7
[junit4] 2> 445571 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [n:127.0.0.1:38021_zki%2Fh ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@77971aa7
[junit4] 2> 445572 INFO (TEST-HttpPartitionTest.test-seed#[BB1D69E7C5127864]) [n:127.0.0.1:38021_zki%2Fh ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_BB1D69E7C5127864-001/shard-3-001/cores
[junit4] 2> 445698 INFO (SocketProxy-Acceptor-38021) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=39852,localport=38021], receiveBufferSize:531000
[junit4] 2> 445723 INFO (SocketProxy-Acceptor-38021) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=44225,localport=35104], receiveBufferSize=530904
[junit4] 2> 445732 INFO (qtp1396324898-3317) [n:127.0.0.1:38021_zki%2Fh ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:38021_zki%252Fh&action=ADDREPLICA&collection=collection1&shard=shard2&type=TLOG&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 445739 INFO (OverseerCollectionConfigSetProcessor-98488607440371716-127.0.0.1:52717_zki%2Fh-n_0000000000) [n:127.0.0.1:52717_zki%2Fh ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000006 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 445740 INFO (OverseerThreadFactory-1087-thread-5-processing-n:127.0.0.1:52717_zki%2Fh) [n:127.0.0.1:52717_zki%2Fh ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:38021_zki%2Fh for creating new replica
[junit4] 2> 445742 INFO (SocketProxy-Acceptor-38021) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=39854,localport=38021], receiveBufferSize:531000
[junit4] 2> 445764 INFO (SocketProxy-Acceptor-38021) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=44225,localport=35106], receiveBufferSize=530904
[junit4] 2> 445785 INFO (qtp1396324898-3315) [n:127.0.0.1:38021_zki%2Fh ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_t45&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=TLOG
[junit4] 2> 445786 INFO (qtp1396324898-3315) [n:127.0.0.1:38021_zki%2Fh ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 446046 INFO (zkCallback-349-thread-2-processing-n:127.0.0.1:49734_zki%2Fh) [n:127.0.0.1:49734_zki%2Fh ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 446046 INFO (zkCallback-355-thread-1-processing-n:127.0.0.1:38021_zki%2Fh) [n:127.0.0.1:38021_zki%2Fh ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 446046 INFO (zkCallback-343-thread-1-processing-n:127.0.0.1:52426_zki%2Fh) [n:127.0.0.1:52426_zki%2Fh ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 446815 INFO (qtp1396324898-3315) [n:127.0.0.1:38021_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 446855 INFO (qtp1396324898-3315) [n:127.0.0.1:38021_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.s.IndexSchema [collection1_shard2_replica_t45] Schema name=test
[junit4] 2> 447283 INFO (qtp1396324898-3315) [n:127.0.0.1:38021_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 447354 INFO (qtp1396324898-3315) [n:127.0.0.1:38021_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_t45' using configuration from collection collection1, trusted=true
[junit4] 2> 447357 INFO (qtp1396324898-3315) [n:127.0.0.1:38021_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_t45' (registry 'solr.core.collection1.shard2.replica_t45') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@77971aa7
[junit4] 2> 447368 INFO (qtp1396324898-3315) [n:127.0.0.1:38021_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 447368 INFO (qtp1396324898-3315) [n:127.0.0.1:38021_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.c.SolrCore [[collection1_shard2_replica_t45] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_BB1D69E7C5127864-001/shard-3-001/cores/collection1_shard2_replica_t45], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.HttpPartitionTest_BB1D69E7C5127864-001/shard-3-001/cores/collection1_shard2_replica_t45/data/]
[junit4] 2> 447843 INFO (qtp1396324898-3315) [n:127.0.0.1:38021_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@4e4d8563
[junit4] 2> 447896 WARN (qtp1396324898-3315) [n:127.0.0.1:38021_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 447977 INFO (qtp1396324898-3315) [n:127.0.0.1:38021_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 447977 INFO (qtp1396324898-3315) [n:127.0.0.1:38021_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 447978 INFO (qtp1396324898-3315) [n:127.0.0.1:38021_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 447978 INFO (qtp1396324898-3315) [n:127.0.0.1:38021_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 447980 INFO (qtp1396324898-3315) [n:127.0.0.1:38021_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=14, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 447980 INFO (qtp1396324898-3315) [n:127.0.0.1:38021_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.s.SolrIndexSearcher Opening [Searcher@493fdde4[collection1_shard2_replica_t45] main]
[junit4] 2> 447993 INFO (qtp1396324898-3315) [n:127.0.0.1:38021_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 448006 INFO (qtp1396324898-3315) [n:127.0.0.1:38021_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 448007 INFO (qtp1396324898-3315) [n:127.0.0.1:38021_zki%2Fh c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 448008 I
[...truncated too long message...]
stReplicasInLIRNoLeader-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1720419584
[junit4] 2> 1612809 INFO (TEST-ForceLeaderTest.testReplicasInLIRNoLeader-seed#[BB1D69E7C5127864]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 1612809 INFO (TEST-ForceLeaderTest.testReplicasInLIRNoLeader-seed#[BB1D69E7C5127864]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@637adc3a: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@7499773
[junit4] 2> 1612810 INFO (TEST-ForceLeaderTest.testReplicasInLIRNoLeader-seed#[BB1D69E7C5127864]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 1612811 INFO (TEST-ForceLeaderTest.testReplicasInLIRNoLeader-seed#[BB1D69E7C5127864]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@29901b23: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@109e9b1
[junit4] 2> 1612812 INFO (TEST-ForceLeaderTest.testReplicasInLIRNoLeader-seed#[BB1D69E7C5127864]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 1612812 INFO (TEST-ForceLeaderTest.testReplicasInLIRNoLeader-seed#[BB1D69E7C5127864]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@177ed0a3: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@578d9f80
[junit4] 2> 1612813 INFO (TEST-ForceLeaderTest.testReplicasInLIRNoLeader-seed#[BB1D69E7C5127864]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 1612820 INFO (coreCloseExecutor-4413-thread-1) [n:127.0.0.1:36260_ c:collection1 s:shard1 r:core_node44 x:collection1_shard1_replica_t43] o.a.s.c.SolrCore [collection1_shard1_replica_t43] CLOSING SolrCore org.apache.solr.core.SolrCore@6da992a8
[junit4] 2> 1612820 INFO (coreCloseExecutor-4413-thread-1) [n:127.0.0.1:36260_ c:collection1 s:shard1 r:core_node44 x:collection1_shard1_replica_t43] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1.shard1.replica_t43, tag=1839829672
[junit4] 2> 1612821 INFO (coreCloseExecutor-4413-thread-1) [n:127.0.0.1:36260_ c:collection1 s:shard1 r:core_node44 x:collection1_shard1_replica_t43] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@18fb6b06: rootName = null, domain = solr.core.collection1.shard1.replica_t43, service url = null, agent id = null] for registry solr.core.collection1.shard1.replica_t43 / com.codahale.metrics.MetricRegistry@5e877c52
[junit4] 2> 1612826 INFO (coreCloseExecutor-4413-thread-1) [n:127.0.0.1:36260_ c:collection1 s:shard1 r:core_node44 x:collection1_shard1_replica_t43] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard1.leader, tag=1839829672
[junit4] 2> 1612828 INFO (TEST-ForceLeaderTest.testReplicasInLIRNoLeader-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.Overseer Overseer (id=98488682242899980-127.0.0.1:36260_-n_0000000002) closing
[junit4] 2> 1612828 INFO (OverseerStateUpdate-98488682242899980-127.0.0.1:36260_-n_0000000002) [n:127.0.0.1:36260_ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:36260_
[junit4] 2> 1614334 WARN (zkCallback-1670-thread-2-processing-n:127.0.0.1:36260_) [n:127.0.0.1:36260_ ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 1614335 INFO (TEST-ForceLeaderTest.testReplicasInLIRNoLeader-seed#[BB1D69E7C5127864]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@62bde143{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 1614335 INFO (TEST-ForceLeaderTest.testReplicasInLIRNoLeader-seed#[BB1D69E7C5127864]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1f2476a5{/,null,UNAVAILABLE}
[junit4] 2> 1614336 INFO (TEST-ForceLeaderTest.testReplicasInLIRNoLeader-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.ChaosMonkey monkey: stop jetty! 41244
[junit4] 2> 1614336 ERROR (TEST-ForceLeaderTest.testReplicasInLIRNoLeader-seed#[BB1D69E7C5127864]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1614338 INFO (TEST-ForceLeaderTest.testReplicasInLIRNoLeader-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:34292 34292
[junit4] 2> 1614432 INFO (Thread-2983) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:34292 34292
[junit4] 2> 1614433 WARN (Thread-2983) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 6 /solr/aliases.json
[junit4] 2> 4 /solr/security.json
[junit4] 2> 4 /solr/configs/conf1
[junit4] 2> 3 /solr/collections/forceleader_test_collection/state.json
[junit4] 2> 3 /solr/collections/collection1/state.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 6 /solr/clusterstate.json
[junit4] 2> 6 /solr/clusterprops.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 6 /solr/live_nodes
[junit4] 2> 6 /solr/collections
[junit4] 2> 3 /solr/overseer/queue
[junit4] 2> 3 /solr/overseer/collection-queue-work
[junit4] 2>
[junit4] 2> 1614433 WARN (TEST-ForceLeaderTest.testReplicasInLIRNoLeader-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.SocketProxy Closing 0 connections to: http://127.0.0.1:41244/, target: http://127.0.0.1:40176/
[junit4] 2> 1614433 WARN (TEST-ForceLeaderTest.testReplicasInLIRNoLeader-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.SocketProxy Closing 6 connections to: http://127.0.0.1:34618/, target: http://127.0.0.1:57011/
[junit4] 2> 1614434 WARN (TEST-ForceLeaderTest.testReplicasInLIRNoLeader-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.SocketProxy Closing 3 connections to: http://127.0.0.1:36260/, target: http://127.0.0.1:34537/
[junit4] 2> 1614434 WARN (TEST-ForceLeaderTest.testReplicasInLIRNoLeader-seed#[BB1D69E7C5127864]) [ ] o.a.s.c.SocketProxy Closing 12 connections to: http://127.0.0.1:60844/, target: http://127.0.0.1:55081/
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=ForceLeaderTest -Dtests.method=testReplicasInLIRNoLeader -Dtests.seed=BB1D69E7C5127864 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=ja-JP -Dtests.timezone=America/St_Kitts -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] FAILURE 35.3s J0 | ForceLeaderTest.testReplicasInLIRNoLeader <<<
[junit4] > Throwable #1: java.lang.AssertionError: Doc with id=1 not found in http://127.0.0.1:60844/forceleader_test_collection due to: Path not found: /id; rsp={doc=null}
[junit4] > at __randomizedtesting.SeedInfo.seed([BB1D69E7C5127864:5D8A5D27FC908105]:0)
[junit4] > at org.apache.solr.cloud.HttpPartitionTest.assertDocExists(HttpPartitionTest.java:603)
[junit4] > at org.apache.solr.cloud.HttpPartitionTest.assertDocsExistInAllReplicas(HttpPartitionTest.java:556)
[junit4] > at org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader(ForceLeaderTest.java:142)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_BB1D69E7C5127864-001
[junit4] 2> Aug 15, 2017 5:27:42 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarity(queryNorm=false): {}, locale=ja-JP, timezone=America/St_Kitts
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=112309344,total=523239424
[junit4] 2> NOTE: All tests run in this JVM: [TestManagedSchemaAPI, DeleteInactiveReplicaTest, TestUseDocValuesAsStored2, TestGeoJSONResponseWriter, TestConfigSetsAPI, TestFileDictionaryLookup, SearchHandlerTest, SimpleCollectionCreateDeleteTest, TestLeaderElectionWithEmptyReplica, SolrIndexMetricsTest, DistributedQueryElevationComponentTest, FieldMutatingUpdateProcessorTest, TestManagedResourceStorage, TestReplicationHandler, EchoParamsTest, DistributedSpellCheckComponentTest, RollingRestartTest, TestPerFieldSimilarityWithDefaultOverride, TestCollectionAPIs, TestDocumentBuilder, TestRestoreCore, TestClusterProperties, BlockJoinFacetSimpleTest, CdcrUpdateLogTest, TestCustomSort, TestLocalFSCloudBackupRestore, TestSchemaSimilarityResource, TestPivotHelperCode, ForceLeaderTest]
[junit4] Completed [248/729 (3!)] on J0 in 101.96s, 3 tests, 1 failure, 1 skipped <<< FAILURES!
[...truncated 44933 lines...]
[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 29 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/29/
6 tests failed.
FAILED: org.apache.solr.TestDistributedSearch.test
Error Message:
Captured an uncaught exception in thread: Thread[id=35810, name=Thread-32472, state=RUNNABLE, group=TGRP-TestDistributedSearch]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=35810, name=Thread-32472, state=RUNNABLE, group=TGRP-TestDistributedSearch]
at __randomizedtesting.SeedInfo.seed([97C3A76F18A3F955:1F9798B5B65F94AD]:0)
Caused by: java.lang.AssertionError: Expected to find shardAddress in the up shard info: {error=org.apache.solr.client.solrj.SolrServerException: Time allowed to handle this request exceeded,trace=org.apache.solr.client.solrj.SolrServerException: Time allowed to handle this request exceeded
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:460)
at org.apache.solr.handler.component.HttpShardHandlerFactory.makeLoadBalancedRequest(HttpShardHandlerFactory.java:273)
at org.apache.solr.handler.component.HttpShardHandler.lambda$submit$0(HttpShardHandler.java:175)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
,time=9}
at __randomizedtesting.SeedInfo.seed([97C3A76F18A3F955]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.TestDistributedSearch.comparePartialResponses(TestDistributedSearch.java:1191)
at org.apache.solr.TestDistributedSearch$1.run(TestDistributedSearch.java:1147)
FAILED: org.apache.solr.cloud.hdfs.HdfsBasicDistributedZkTest.test
Error Message:
Error from server at http://127.0.0.1:35444/b_f/e: ADDREPLICA failed to create replica
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:35444/b_f/e: ADDREPLICA failed to create replica
at __randomizedtesting.SeedInfo.seed([97C3A76F18A3F955:1F9798B5B65F94AD]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:627)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1121)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:862)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:793)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195)
at org.apache.solr.cloud.BasicDistributedZkTest.testANewCollectionInOneInstanceWithManualShardAssignement(BasicDistributedZkTest.java:801)
at org.apache.solr.cloud.BasicDistributedZkTest.test(BasicDistributedZkTest.java:374)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsBasicDistributedZkTest
Error Message:
3 threads leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsBasicDistributedZkTest: 1) Thread[id=7035, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HdfsBasicDistributedZkTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=7043, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HdfsBasicDistributedZkTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=7054, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HdfsBasicDistributedZkTest] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 3 threads leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsBasicDistributedZkTest:
1) Thread[id=7035, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HdfsBasicDistributedZkTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=7043, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HdfsBasicDistributedZkTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=7054, name=Connection evictor, state=TIMED_WAITING, group=TGRP-HdfsBasicDistributedZkTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([97C3A76F18A3F955]:0)
FAILED: org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader
Error Message:
Doc with id=1 not found in http://127.0.0.1:49630/forceleader_test_collection due to: Path not found: /id; rsp={doc=null}
Stack Trace:
java.lang.AssertionError: Doc with id=1 not found in http://127.0.0.1:49630/forceleader_test_collection due to: Path not found: /id; rsp={doc=null}
at __randomizedtesting.SeedInfo.seed([97C3A76F18A3F955:715493AF21210034]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.HttpPartitionTest.assertDocExists(HttpPartitionTest.java:603)
at org.apache.solr.cloud.HttpPartitionTest.assertDocsExistInAllReplicas(HttpPartitionTest.java:556)
at org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader(ForceLeaderTest.java:142)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_dv
Error Message:
java.lang.RuntimeException: Error from server at http://127.0.0.1:39298/solr/test_col_shard3_replica_n8: Failed synchronous update on shard StdNode: http://127.0.0.1:45644/solr/test_col_shard3_replica_n10/ update: org.apache.solr.client.solrj.request.UpdateRequest@4b3e4597
Stack Trace:
java.util.concurrent.ExecutionException: java.lang.RuntimeException: Error from server at http://127.0.0.1:39298/solr/test_col_shard3_replica_n8: Failed synchronous update on shard StdNode: http://127.0.0.1:45644/solr/test_col_shard3_replica_n10/ update: org.apache.solr.client.solrj.request.UpdateRequest@4b3e4597
at __randomizedtesting.SeedInfo.seed([97C3A76F18A3F955:A1D7C52992FEC344]:0)
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.checkField(TestStressCloudBlindAtomicUpdates.java:283)
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_dv(TestStressCloudBlindAtomicUpdates.java:195)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: Error from server at http://127.0.0.1:39298/solr/test_col_shard3_replica_n8: Failed synchronous update on shard StdNode: http://127.0.0.1:45644/solr/test_col_shard3_replica_n10/ update: org.apache.solr.client.solrj.request.UpdateRequest@4b3e4597
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.run(TestStressCloudBlindAtomicUpdates.java:411)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
Caused by: org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from server at http://127.0.0.1:39298/solr/test_col_shard3_replica_n8: Failed synchronous update on shard StdNode: http://127.0.0.1:45644/solr/test_col_shard3_replica_n10/ update: org.apache.solr.client.solrj.request.UpdateRequest@4b3e4597
at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:539)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:993)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:862)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:793)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195)
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.doRandomAtomicUpdate(TestStressCloudBlindAtomicUpdates.java:370)
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.run(TestStressCloudBlindAtomicUpdates.java:399)
... 6 more
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:39298/solr/test_col_shard3_replica_n8: Failed synchronous update on shard StdNode: http://127.0.0.1:45644/solr/test_col_shard3_replica_n10/ update: org.apache.solr.client.solrj.request.UpdateRequest@4b3e4597
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:627)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:516)
... 5 more
FAILED: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test
Error Message:
expected:<2> but was:<3>
Stack Trace:
java.lang.AssertionError: expected:<2> but was:<3>
at __randomizedtesting.SeedInfo.seed([97C3A76F18A3F955:1F9798B5B65F94AD]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.assertSliceAndReplicaCount(SharedFSAutoReplicaFailoverTest.java:403)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.testBasics(SharedFSAutoReplicaFailoverTest.java:248)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test(SharedFSAutoReplicaFailoverTest.java:136)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 11623 lines...]
[junit4] Suite: org.apache.solr.TestDistributedSearch
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_97C3A76F18A3F955-001/init-core-data-001
[junit4] 2> 358560 WARN (SUITE-TestDistributedSearch-seed#[97C3A76F18A3F955]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=2 numCloses=2
[junit4] 2> 358560 INFO (SUITE-TestDistributedSearch-seed#[97C3A76F18A3F955]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 358561 INFO (SUITE-TestDistributedSearch-seed#[97C3A76F18A3F955]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-9061)
[junit4] 2> 358561 INFO (SUITE-TestDistributedSearch-seed#[97C3A76F18A3F955]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 358561 INFO (SUITE-TestDistributedSearch-seed#[97C3A76F18A3F955]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /lfbf/a
[junit4] 2> 358654 INFO (TEST-TestDistributedSearch.test-seed#[97C3A76F18A3F955]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_97C3A76F18A3F955-001/tempDir-001/control/cores/collection1
[junit4] 2> 358655 INFO (TEST-TestDistributedSearch.test-seed#[97C3A76F18A3F955]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 358656 INFO (TEST-TestDistributedSearch.test-seed#[97C3A76F18A3F955]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6cbf38b2{/lfbf/a,null,AVAILABLE}
[junit4] 2> 358658 INFO (TEST-TestDistributedSearch.test-seed#[97C3A76F18A3F955]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@6f47b3e7{HTTP/1.1,[http/1.1]}{127.0.0.1:49461}
[junit4] 2> 358658 INFO (TEST-TestDistributedSearch.test-seed#[97C3A76F18A3F955]) [ ] o.e.j.s.Server Started @361421ms
[junit4] 2> 358658 INFO (TEST-TestDistributedSearch.test-seed#[97C3A76F18A3F955]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/lfbf/a, hostPort=49461, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_97C3A76F18A3F955-001/tempDir-001/control/cores}
[junit4] 2> 358658 ERROR (TEST-TestDistributedSearch.test-seed#[97C3A76F18A3F955]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 358658 INFO (TEST-TestDistributedSearch.test-seed#[97C3A76F18A3F955]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.1.0
[junit4] 2> 358658 INFO (TEST-TestDistributedSearch.test-seed#[97C3A76F18A3F955]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 358659 INFO (TEST-TestDistributedSearch.test-seed#[97C3A76F18A3F955]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 358659 INFO (TEST-TestDistributedSearch.test-seed#[97C3A76F18A3F955]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-14T14:12:28.413Z
[junit4] 2> 358659 INFO (TEST-TestDistributedSearch.test-seed#[97C3A76F18A3F955]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_97C3A76F18A3F955-001/tempDir-001/control/solr.xml
[junit4] 2> 358670 INFO (TEST-TestDistributedSearch.test-seed#[97C3A76F18A3F955]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@3878351e, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 358713 INFO (TEST-TestDistributedSearch.test-seed#[97C3A76F18A3F955]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3878351e
[junit4] 2> 358716 INFO (TEST-TestDistributedSearch.test-seed#[97C3A76F18A3F955]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3878351e
[junit4] 2> 358716 INFO (TEST-TestDistributedSearch.test-seed#[97C3A76F18A3F955]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3878351e
[junit4] 2> 358721 INFO (TEST-TestDistributedSearch.test-seed#[97C3A76F18A3F955]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_97C3A76F18A3F955-001/tempDir-001/control/cores
[junit4] 2> 358721 INFO (TEST-TestDistributedSearch.test-seed#[97C3A76F18A3F955]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 358722 INFO (coreLoadExecutor-1437-thread-1) [ x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
[junit4] 2> 358771 INFO (coreLoadExecutor-1437-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 358789 INFO (coreLoadExecutor-1437-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 358888 INFO (coreLoadExecutor-1437-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 358894 INFO (coreLoadExecutor-1437-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from configset /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1, trusted=true
[junit4] 2> 358894 INFO (coreLoadExecutor-1437-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3878351e
[junit4] 2> 358894 INFO (coreLoadExecutor-1437-thread-1) [ x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 358895 INFO (coreLoadExecutor-1437-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_97C3A76F18A3F955-001/tempDir-001/control/cores/collection1/data/]
[junit4] 2> 358896 INFO (coreLoadExecutor-1437-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=37, maxMergeAtOnceExplicit=29, maxMergedSegmentMB=87.85546875, floorSegmentMB=1.0546875, forceMergeDeletesPctAllowed=0.8208835870824371, segmentsPerTier=33.0, maxCFSSegmentSizeMB=1.6123046875, noCFSRatio=0.49313723117389097
[junit4] 2> 358932 INFO (coreLoadExecutor-1437-thread-1) [ x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 358932 INFO (coreLoadExecutor-1437-thread-1) [ x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 358933 INFO (coreLoadExecutor-1437-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 358933 INFO (coreLoadExecutor-1437-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 358933 INFO (coreLoadExecutor-1437-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@134f227a
[junit4] 2> 358934 INFO (coreLoadExecutor-1437-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@44efc8b3[collection1] main]
[junit4] 2> 358934 INFO (coreLoadExecutor-1437-thread-1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
[junit4] 2> 358943 INFO (coreLoadExecutor-1437-thread-1) [ x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4] 2> 358969 INFO (coreLoadExecutor-1437-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 358969 INFO (searcherExecutor-1438-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
[junit4] 2> 358970 INFO (coreLoadExecutor-1437-thread-1) [ x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575716072953217024
[junit4] 2> 358970 INFO (searcherExecutor-1438-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
[junit4] 2> 358970 INFO (searcherExecutor-1438-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
[junit4] 2> 358970 INFO (searcherExecutor-1438-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
[junit4] 2> 358970 INFO (searcherExecutor-1438-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
[junit4] 2> 358970 INFO (searcherExecutor-1438-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
[junit4] 2> 358970 INFO (searcherExecutor-1438-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
[junit4] 2> 358971 INFO (searcherExecutor-1438-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
[junit4] 2> 358971 INFO (searcherExecutor-1438-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
[junit4] 2> 358977 INFO (searcherExecutor-1438-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@44efc8b3[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 359545 INFO (TEST-TestDistributedSearch.test-seed#[97C3A76F18A3F955]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_97C3A76F18A3F955-001/tempDir-001/shard0/cores/collection1
[junit4] 2> 359570 INFO (TEST-TestDistributedSearch.test-seed#[97C3A76F18A3F955]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 359571 INFO (TEST-TestDistributedSearch.test-seed#[97C3A76F18A3F955]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@337342ff{/lfbf/a,null,AVAILABLE}
[junit4] 2> 359571 INFO (TEST-TestDistributedSearch.test-seed#[97C3A76F18A3F955]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@76ebebaa{HTTP/1.1,[http/1.1]}{127.0.0.1:50033}
[junit4] 2> 359571 INFO (TEST-TestDistributedSearch.test-seed#[97C3A76F18A3F955]) [ ] o.e.j.s.Server Started @362334ms
[junit4] 2> 359571 INFO (TEST-TestDistributedSearch.test-seed#[97C3A76F18A3F955]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/lfbf/a, hostPort=50033, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_97C3A76F18A3F955-001/tempDir-001/shard0/cores}
[junit4] 2> 359571 ERROR (TEST-TestDistributedSearch.test-seed#[97C3A76F18A3F955]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 359571 INFO (TEST-TestDistributedSearch.test-seed#[97C3A76F18A3F955]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.1.0
[junit4] 2> 359571 INFO (TEST-TestDistributedSearch.test-seed#[97C3A76F18A3F955]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 359571 INFO (TEST-TestDistributedSearch.test-seed#[97C3A76F18A3F955]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 359571 INFO (TEST-TestDistributedSearch.test-seed#[97C3A76F18A3F955]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-14T14:12:29.325Z
[junit4] 2> 359572 INFO (TEST-TestDistributedSearch.test-seed#[97C3A76F18A3F955]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_97C3A76F18A3F955-001/tempDir-001/shard0/solr.xml
[junit4] 2> 359588 INFO (TEST-TestDistributedSearch.test-seed#[97C3A76F18A3F955]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@3878351e, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 359660 INFO (TEST-TestDistributedSearch.test-seed#[97C3A76F18A3F955]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3878351e
[junit4] 2> 359664 INFO (TEST-TestDistributedSearch.test-seed#[97C3A76F18A3F955]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3878351e
[junit4] 2> 359664 INFO (TEST-TestDistributedSearch.test-seed#[97C3A76F18A3F955]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3878351e
[junit4] 2> 359685 INFO (TEST-TestDistributedSearch.test-seed#[97C3A76F18A3F955]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_97C3A76F18A3F955-001/tempDir-001/shard0/cores
[junit4] 2> 359685 INFO (TEST-TestDistributedSearch.test-seed#[97C3A76F18A3F955]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 359697 INFO (coreLoadExecutor-1448-thread-1) [ x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
[junit4] 2> 359745 INFO (coreLoadExecutor-1448-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 359780 INFO (coreLoadExecutor-1448-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 360018 INFO (coreLoadExecutor-1448-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 360029 INFO (coreLoadExecutor-1448-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from configset /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1, trusted=true
[junit4] 2> 360029 INFO (coreLoadExecutor-1448-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3878351e
[junit4] 2> 360029 INFO (coreLoadExecutor-1448-thread-1) [ x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 360029 INFO (coreLoadExecutor-1448-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.TestDistributedSearch_97C3A76F18A3F955-001/tempDir-001/shard0/cores/collection1/data/]
[junit4] 2> 360031 INFO (coreLoadExecutor-1448-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=37, maxMergeAtOnceExplicit=29, maxMergedSegmentMB=87.85546875, floorSegmentMB=1.0546875, forceMergeDeletesPctAllowed=0.8208835870824371, segmentsPerTier=33.0, maxCFSSegmentSizeMB=1.6123046875, noCFSRatio=0.49313723117389097
[junit4] 2> 360071 INFO (coreLoadExecutor-1448-thread-1) [ x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 360071 INFO (coreLoadExecutor-1448-thread-1) [ x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 360071 INFO (coreLoadExecutor-1448-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 360071 INFO (coreLoadExecutor-1448-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 360072 INFO (coreLoadExecutor-1448-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@7135f654
[junit4] 2> 360072 INFO (coreLoadExecutor-1448-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@6b4a9d5f[collection1] main]
[junit4] 2> 360073 INFO (coreLoadExecutor-1448-thread-1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
[junit4] 2> 360091 INFO (coreLoadExecutor-1448-thread-1) [ x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4] 2> 360126 INFO (coreLoadExecutor-1448-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 360127 INFO (searcherExecutor-1449-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
[junit4] 2> 360127 INFO (searcherExecutor-1449-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
[junit4] 2> 360127 INFO (searcherExecutor-1449-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
[junit4] 2> 360127 INFO (searcherExecutor-1449-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
[junit4] 2> 360127 INFO (searcherExecutor-1449-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
[junit4] 2> 360127 INFO (searcherExecutor-1449-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
[junit4] 2> 360127 INFO (searcherExecutor-1449-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
[junit4] 2> 360128 INFO (searcherExecutor-1449-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
[junit4] 2> 360128 INFO (searcherExecutor-1449-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
[junit4] 2> 360132 INFO (searcherExecutor-1449-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@6b4a9d5f[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 360132 INFO (coreLoadExecutor-1448-thread-1) [ x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575716074171662336
[junit4] 2> 360235 INFO (TEST-TestDistributedSearch.test-seed#[97C3A76F18A3F955]) [ ] o.a.s.SolrTestCaseJ4 ###Starting test
[junit4] 2> 360254 INFO (qtp500765633-4087) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{deleteByQuery=*:* (-1575716074296442880)} 0 2
[junit4] 2> 360303 INFO (qtp1699784955-4104) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{deleteByQuery=*:* (-1575716074349920256)} 0 1
[junit4] 2> 360312 INFO (qtp500765633-4088) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[1 (1575716074352017408)]} 0 7
[junit4] 2> 360319 INFO (qtp1699784955-4105) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[1 (1575716074360406016)]} 0 6
[junit4] 2> 360321 INFO (qtp500765633-4089) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[2 (1575716074368794624)]} 0 1
[junit4] 2> 360331 INFO (qtp1699784955-4106) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[2 (1575716074369843200)]} 0 9
[junit4] 2> 360332 INFO (qtp500765633-4090) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[3 (1575716074380328960)]} 0 0
[junit4] 2> 360333 INFO (qtp1699784955-4107) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[3 (1575716074381377536)]} 0 0
[junit4] 2> 360334 INFO (qtp500765633-4090) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[4 (1575716074382426112)]} 0 0
[junit4] 2> 360334 INFO (qtp1699784955-4107) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[4 (1575716074383474688)]} 0 0
[junit4] 2> 360335 INFO (qtp500765633-4090) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[5 (1575716074384523264)]} 0 0
[junit4] 2> 360337 INFO (qtp1699784955-4101) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[5 (1575716074385571840)]} 0 0
[junit4] 2> 360342 INFO (qtp500765633-4087) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[6 (1575716074390814720)]} 0 0
[junit4] 2> 360343 INFO (qtp1699784955-4101) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[6 (1575716074391863296)]} 0 0
[junit4] 2> 360343 INFO (qtp500765633-4088) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[7 (1575716074392911872)]} 0 0
[junit4] 2> 360344 INFO (qtp1699784955-4101) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[7 (1575716074393960448)]} 0 0
[junit4] 2> 360345 INFO (qtp500765633-4088) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[8 (1575716074395009024)]} 0 0
[junit4] 2> 360346 INFO (qtp1699784955-4101) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[8 (1575716074395009024)]} 0 0
[junit4] 2> 360347 INFO (qtp500765633-4088) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[9 (1575716074396057600)]} 0 0
[junit4] 2> 360347 INFO (qtp1699784955-4101) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[9 (1575716074397106176)]} 0 0
[junit4] 2> 360348 INFO (qtp500765633-4088) [ x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1575716074398154752,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 360348 INFO (qtp500765633-4088) [ x:collection1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@37ed00fb commitCommandVersion:1575716074398154752
[junit4] 2> 360390 INFO (qtp500765633-4088) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2042f013[collection1] main]
[junit4] 2> 360391 INFO (searcherExecutor-1438-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2042f013[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.1.0):c9)))}
[junit4] 2> 360391 INFO (qtp500765633-4088) [ x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 360391 INFO (qtp500765633-4088) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 43
[junit4] 2> 360392 INFO (qtp1699784955-4107) [ x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1575716074444292096,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 360392 INFO (qtp1699784955-4107) [ x:collection1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@49e7cd6b commitCommandVersion:1575716074444292096
[junit4] 2> 360412 INFO (qtp1699784955-4107) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@290c6445[collection1] main]
[junit4] 2> 360414 INFO (searcherExecutor-1449-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@290c6445[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.1.0):c9)))}
[junit4] 2> 360414 INFO (qtp1699784955-4107) [ x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 360414 INFO (qtp1699784955-4107) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 22
[junit4] 2> 360447 INFO (qtp500765633-4090) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[10 (1575716074474700800)]} 0 26
[junit4] 2> 360449 INFO (qtp1699784955-4104) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[10 (1575716074503012352)]} 0 1
[junit4] 2> 360451 INFO (qtp500765633-4087) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[11 (1575716074505109504)]} 0 0
[junit4] 2> 360454 INFO (qtp1699784955-4105) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[11 (1575716074506158080)]} 0 2
[junit4] 2> 360454 INFO (qtp500765633-4087) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[12 (1575716074509303808)]} 0 0
[junit4] 2> 360455 INFO (qtp1699784955-4106) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[12 (1575716074510352384)]} 0 0
[junit4] 2> 360456 INFO (qtp500765633-4087) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[13 (1575716074510352384)]} 0 0
[junit4] 2> 360456 INFO (qtp1699784955-4106) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[13 (1575716074511400960)]} 0 0
[junit4] 2> 360458 INFO (qtp500765633-4087) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[1001 (1575716074512449536)]} 0 1
[junit4] 2> 360459 INFO (qtp1699784955-4106) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[1001 (1575716074514546688)]} 0 0
[junit4] 2> 360465 INFO (qtp500765633-4089) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[14 (1575716074520838144)]} 0 0
[junit4] 2> 360466 INFO (qtp1699784955-4107) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[14 (1575716074521886720)]} 0 0
[junit4] 2> 360473 INFO (qtp500765633-4089) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[15 (1575716074529226752)]} 0 0
[junit4] 2> 360474 INFO (qtp1699784955-4107) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[15 (1575716074530275328)]} 0 0
[junit4] 2> 360477 INFO (qtp500765633-4089) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[16 (1575716074533421056)]} 0 0
[junit4] 2> 360478 INFO (qtp1699784955-4107) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[16 (1575716074534469632)]} 0 0
[junit4] 2> 360479 INFO (qtp500765633-4088) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[17 (1575716074534469632)]} 0 0
[junit4] 2> 360480 INFO (qtp1699784955-4108) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[17 (1575716074535518208)]} 0 0
[junit4] 2> 360480 INFO (qtp500765633-4088) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[100 (1575716074536566784)]} 0 0
[junit4] 2> 360481 INFO (qtp1699784955-4108) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[100 (1575716074537615360)]} 0 0
[junit4] 2> 360482 INFO (qtp500765633-4088) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[101 (1575716074537615360)]} 0 0
[junit4] 2> 360482 INFO (qtp1699784955-4108) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[101 (1575716074538663936)]} 0 0
[junit4] 2> 360483 INFO (qtp500765633-4088) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[102 (1575716074539712512)]} 0 0
[junit4] 2> 360484 INFO (qtp1699784955-4108) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[102 (1575716074539712512)]} 0 0
[junit4] 2> 360484 INFO (qtp500765633-4088) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[103 (1575716074540761088)]} 0 0
[junit4] 2> 360485 INFO (qtp1699784955-4108) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[103 (1575716074541809664)]} 0 0
[junit4] 2> 360485 INFO (qtp500765633-4088) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[104 (1575716074541809664)]} 0 0
[junit4] 2> 360486 INFO (qtp1699784955-4108) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[104 (1575716074542858240)]} 0 0
[junit4] 2> 360487 INFO (qtp500765633-4088) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[105 (1575716074542858240)]} 0 0
[junit4] 2> 360487 INFO (qtp1699784955-4108) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[105 (1575716074543906816)]} 0 0
[junit4] 2> 360488 INFO (qtp500765633-4088) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[106 (1575716074544955392)]} 0 0
[junit4] 2> 360489 INFO (qtp1699784955-4106) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[106 (1575716074544955392)]} 0 0
[junit4] 2> 360489 INFO (qtp500765633-4088) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[107 (1575716074546003968)]} 0 0
[junit4] 2> 360490 INFO (qtp1699784955-4106) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[107 (1575716074547052544)]} 0 0
[junit4] 2> 360491 INFO (qtp500765633-4090) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[108 (1575716074547052544)]} 0 0
[junit4] 2> 360491 INFO (qtp1699784955-4106) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[108 (1575716074548101120)]} 0 0
[junit4] 2> 360492 INFO (qtp500765633-4090) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[109 (1575716074549149696)]} 0 0
[junit4] 2> 360493 INFO (qtp1699784955-4106) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[109 (1575716074549149696)]} 0 0
[junit4] 2> 360493 INFO (qtp500765633-4090) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[110 (1575716074550198272)]} 0 0
[junit4] 2> 360494 INFO (qtp1699784955-4106) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[110 (1575716074551246848)]} 0 0
[junit4] 2> 360495 INFO (qtp500765633-4090) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[111 (1575716074551246848)]} 0 0
[junit4] 2> 360495 INFO (qtp1699784955-4106) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[111 (1575716074552295424)]} 0 0
[junit4] 2> 360496 INFO (qtp500765633-4090) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[112 (1575716074552295424)]} 0 0
[junit4] 2> 360496 INFO (qtp1699784955-4106) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[112 (1575716074553344000)]} 0 0
[junit4] 2> 360497 INFO (qtp500765633-4090) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[113 (1575716074554392576)]} 0 0
[junit4] 2> 360498 INFO (qtp1699784955-4106) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[113 (1575716074554392576)]} 0 0
[junit4] 2> 360498 INFO (qtp500765633-4090) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[114 (1575716074555441152)]} 0 0
[junit4] 2> 360499 INFO (qtp1699784955-4106) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[114 (1575716074556489728)]} 0 0
[junit4] 2> 360500 INFO (qtp500765633-4090) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[115 (1575716074556489728)]} 0 0
[junit4] 2> 360500 INFO (qtp1699784955-4101) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[115 (1575716074557538304)]} 0 0
[junit4] 2> 360501 INFO (qtp500765633-4090) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[116 (1575716074558586880)]} 0 0
[junit4] 2> 360501 INFO (qtp1699784955-4101) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[116 (1575716074558586880)]} 0 0
[junit4] 2> 360502 INFO (qtp500765633-4087) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[117 (1575716074559635456)]} 0 0
[junit4] 2> 360509 INFO (qtp1699784955-4101) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[117 (1575716074566975488)]} 0 0
[junit4] 2> 360510 INFO (qtp500765633-4087) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[118 (1575716074568024064)]} 0 0
[junit4] 2> 360511 INFO (qtp1699784955-4101) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[118 (1575716074568024064)]} 0 0
[junit4] 2> 360511 INFO (qtp500765633-4087) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[119 (1575716074569072640)]} 0 0
[junit4] 2> 360512 INFO (qtp1699784955-4101) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[119 (1575716074570121216)]} 0 0
[junit4] 2> 360513 INFO (qtp500765633-4087) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[120 (1575716074570121216)]} 0 0
[junit4] 2> 360513 INFO (qtp1699784955-4101) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[120 (1575716074571169792)]} 0 0
[junit4] 2> 360514 INFO (qtp500765633-4087) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[121 (1575716074572218368)]} 0 0
[junit4] 2> 360514 INFO (qtp1699784955-4101) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[121 (1575716074572218368)]} 0 0
[junit4] 2> 360515 INFO (qtp500765633-4087) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[122 (1575716074573266944)]} 0 0
[junit4] 2> 360516 INFO (qtp1699784955-4101) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[122 (1575716074573266944)]} 0 0
[junit4] 2> 360516 INFO (qtp500765633-4087) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[123 (1575716074574315520)]} 0 0
[junit4] 2> 360517 INFO (qtp1699784955-4105) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[123 (1575716074575364096)]} 0 0
[junit4] 2> 360518 INFO (qtp500765633-4087) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[124 (1575716074575364096)]} 0 0
[junit4] 2> 360518 INFO (qtp1699784955-4105) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[124 (1575716074576412672)]} 0 0
[junit4] 2> 360519 INFO (qtp500765633-4084) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[125 (1575716074577461248)]} 0 0
[junit4] 2> 360520 INFO (qtp1699784955-4105) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[125 (1575716074577461248)]} 0 0
[junit4] 2> 360520 INFO (qtp500765633-4084) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[126 (1575716074578509824)]} 0 0
[junit4] 2> 360521 INFO (qtp1699784955-4105) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[126 (1575716074579558400)]} 0 0
[junit4] 2> 360521 INFO (qtp500765633-4084) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[127 (1575716074579558400)]} 0 0
[junit4] 2> 360530 INFO (qtp1699784955-4105) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[127 (1575716074587947008)]} 0 0
[junit4] 2> 360530 INFO (qtp500765633-4084) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[128 (1575716074588995584)]} 0 0
[junit4] 2> 360531 INFO (qtp1699784955-4105) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[128 (1575716074590044160)]} 0 0
[junit4] 2> 360532 INFO (qtp500765633-4084) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[129 (1575716074590044160)]} 0 0
[junit4] 2> 360532 INFO (qtp1699784955-4105) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[129 (1575716074591092736)]} 0 0
[junit4] 2> 360533 INFO (qtp500765633-4084) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[130 (1575716074592141312)]} 0 0
[junit4] 2> 360542 INFO (qtp1699784955-4105) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[130 (1575716074600529920)]} 0 0
[junit4] 2> 360542 INFO (qtp500765633-4084) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[131 (1575716074601578496)]} 0 0
[junit4] 2> 360543 INFO (qtp1699784955-4105) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[131 (1575716074602627072)]} 0 0
[junit4] 2> 360544 INFO (qtp500765633-4084) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[132 (1575716074602627072)]} 0 0
[junit4] 2> 360544 INFO (qtp1699784955-4105) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[132 (1575716074603675648)]} 0 0
[junit4] 2> 360545 INFO (qtp500765633-4084) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[133 (1575716074604724224)]} 0 0
[junit4] 2> 360545 INFO (qtp1699784955-4105) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[133 (1575716074604724224)]} 0 0
[junit4] 2> 360546 INFO (qtp500765633-4084) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[134 (1575716074605772800)]} 0 0
[junit4] 2> 360547 INFO (qtp1699784955-4105) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[134 (1575716074605772800)]} 0 0
[junit4] 2> 360547 INFO (qtp500765633-4084) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[135 (1575716074606821376)]} 0 0
[junit4] 2> 360548 INFO (qtp1699784955-4105) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[135 (1575716074607869952)]} 0 0
[junit4] 2> 360549 INFO (qtp500765633-4084) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[136 (1575716074607869952)]} 0 0
[junit4] 2> 360549 INFO (qtp1699784955-4104) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[136 (1575716074608918528)]} 0 0
[junit4] 2> 360550 INFO (qtp500765633-4084) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[137 (1575716074609967104)]} 0 0
[junit4] 2> 360550 INFO (qtp1699784955-4104) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[137 (1575716074609967104)]} 0 0
[junit4] 2> 360551 INFO (qtp500765633-4084) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[138 (1575716074611015680)]} 0 0
[junit4] 2> 360557 INFO (qtp1699784955-4104) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[138 (1575716074617307136)]} 0 0
[junit4] 2> 360558 INFO (qtp500765633-4084) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[139 (1575716074618355712)]} 0 0
[junit4] 2> 360559 INFO (qtp1699784955-4104) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[139 (1575716074618355712)]} 0 0
[junit4] 2> 360559 INFO (qtp500765633-4084) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[140 (1575716074619404288)]} 0 0
[junit4] 2> 360560 INFO (qtp1699784955-4104) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[140 (1575716074620452864)]} 0 0
[junit4] 2> 360560 INFO (qtp500765633-4089) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[141 (1575716074620452864)]} 0 0
[junit4] 2> 360561 INFO (qtp1699784955-4104) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[141 (1575716074621501440)]} 0 0
[junit4] 2> 360562 INFO (qtp500765633-4089) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[142 (1575716074621501440)]} 0 0
[junit4] 2> 360562 INFO (qtp1699784955-4104) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[142 (1575716074622550016)]} 0 0
[junit4] 2> 360563 INFO (qtp500765633-4089) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[143 (1575716074623598592)]} 0 0
[junit4] 2> 360564 INFO (qtp1699784955-4104) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[143 (1575716074623598592)]} 0 0
[junit4] 2> 360564 INFO (qtp500765633-4089) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[144 (1575716074624647168)]} 0 0
[junit4] 2> 360565 INFO (qtp1699784955-4104) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[144 (1575716074624647168)]} 0 0
[junit4] 2> 360565 INFO (qtp500765633-4089) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[145 (1575716074625695744)]} 0 0
[junit4] 2> 360566 INFO (qtp1699784955-4104) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[145 (1575716074626744320)]} 0 0
[junit4] 2> 360567 INFO (qtp500765633-4089) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[146 (1575716074626744320)]} 0 0
[junit4] 2> 360567 INFO (qtp1699784955-4104) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[146 (1575716074627792896)]} 0 0
[junit4] 2> 360568 INFO (qtp500765633-4089) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[147 (1575716074627792896)]} 0 0
[junit4] 2> 360568 INFO (qtp1699784955-4104) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[147 (1575716074628841472)]} 0 0
[junit4] 2> 360569 INFO (qtp500765633-4089) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[148 (1575716074629890048)]} 0 0
[junit4] 2> 360569 INFO (qtp1699784955-4104) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[148 (1575716074629890048)]} 0 0
[junit4] 2> 360570 INFO (qtp500765633-4089) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[149 (1575716074630938624)]} 0 0
[junit4] 2> 360571 INFO (qtp1699784955-4104) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={wt=javabin&version=2}{add=[149 (1575716074630938624)]} 0 0
[junit4] 2> 360571 INFO (qtp500765633-4089) [ x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1575716074631987200,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 360571 INFO (qtp500765633-4089) [ x:collection1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@37ed00fb commitCommandVersion:1575716074631987200
[junit4] 2> 360596 INFO (qtp500765633-4089) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@62e09f04[collection1] main]
[junit4] 2> 360598 INFO (searcherExecutor-1438-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@62e09f04[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.1.0):c9) Uninverting(_1(7.1.0):c59)))}
[junit4] 2> 360598 INFO (qtp500765633-4089) [ x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 360598 INFO (qtp500765633-4089) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 27
[junit4] 2> 360599 INFO (qtp1699784955-4104) [ x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1575716074661347328,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 360599 INFO (qtp1699784955-4104) [ x:collection1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@49e7cd6b commitCommandVersion:1575716074661347328
[junit4] 2> 360610 INFO (qtp1699784955-4104) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@7d0468c5[collection1] main]
[junit4] 2> 360612 INFO (searcherExecutor-1449-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@7d0468c5[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.1.0):c9) Uninverting(_1(7.1.0):c59)))}
[junit4] 2> 360612 INFO (qtp1699784955-4104) [ x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 360612 INFO (qtp1699784955-4104) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/lfbf/a path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 12
[junit4] 2> 360613 INFO (qtp500765633-4084) [ x:collection1] o.a.s.h.RequestHandlerBase Ignoring exception matching parameter cannot be negative
[junit4] 2> 360613 INFO (qtp500765633-4084) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/lfbf/a path=/select params={q=*&distrib=false&start=-1&wt=javabin&version=2} status=400 QTime=0
[junit4] 2> 360614 INFO (qtp500765633-4084) [ x:collection1] o.a.s.h.RequestHandlerBase Ignoring exception matching parameter cannot be negative
[junit4] 2> 360614 INFO (qtp500765633-4084) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/lfbf/a path=/select params={q=*&distrib=false&start=0&rows=-1&wt=javabin&version=2} status=400 QTime=0
[junit4] 2> 360615 INFO (qtp500765633-4084) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/lfbf/a path=/select params={q=*:*&distrib=false&sort=n_ti1+desc&wt=javabin&version=2} hits=68 status=0 QTime=0
[junit4] 2> 360707 INFO (qtp1699784955-4101) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/lfbf/a path=/select params={df=text&distrib=false&fl=id&shards.purpose=4&start=0&fsv=true&sort=n_ti1+desc&shard.url=[ff01::114]:33332/lfbf/a|http://127.0.0.1:50033/lfbf/a/collection1|[ff01::213]:33332/lfbf/a|[ff01::083]:33332/lfbf/a&rows=10&version=2&q=*:*&NOW=1502719950370&isShard=true&wt=javabin} hits=68 status=0 QTime=0
[junit4] 2> 360712 INFO (qtp1699784955-4101) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/lfbf/a path=/select params={q=*:*&df=text&distrib=false&shards.purpose=64&NOW=1502719950370&ids=11,144,3,15,125,17,117,129,108,10&isShard=true&shard.url=[ff01::114]:33332/lfbf/a|http://127.0.0.1:50033/lfbf/a/collection1|[ff01::213]:33332/lfbf/a|[ff01::083]:33332/lfbf/a&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 360716 INFO (qtp1699784955-4105) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/lfbf/a path=/select params={q=*:*&shards=[ff01::114]:33332/lfbf/a|http://127.0.0.1:50033/lfbf/a/collection1|[ff01::213]:33332/lfbf/a|[ff01::083]:33332/lfbf/a&sort=n_ti1+desc&wt=javabin&version=2} hits=68 status=0 QTime=100
[junit4] 2> 360718 INFO (TEST-TestDistributedSearch.test-seed#[97C3A76F18A3F955]) [ ] o.a.s.BaseDistributedSearchTestCase starting stress...
[junit4] 2> 360764 INFO (qtp1699784955-4102) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/lfbf/a path=/select params={df=text&distrib=false&fl=id&shards.purpose=4&start=0&fsv=true&sort=n_ti1+desc&shard.url=[ff01::114]:33332/lfbf/a|http://127.0.0.1:50033/lfbf/a/collection1|[ff01::213]:33332/lfbf/a|[ff01::083]:33332/lfbf/a&rows=10&version=2&q=*:*&NOW=1502719950487&isShard=true&wt=javabin} hits=68 status=0 QTime=0
[junit4] 2> 360766 INFO (qtp1699784955-4102) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/lfbf/a path=/select params={q=*:*&df=text&distrib=false&shards.purpose=64&NOW=1502719950487&ids=11,144,3,15,125,17,117,129,108,10&isShard=true&shard.url=[ff01::114]:33332/lfbf/a|http://127.0.0.1:50033/lfbf/a/collection1|[ff01::213]:33332/lfbf/a|[ff01::083]:33332/lfbf/a&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 360778 INFO (qtp1699784955-4108) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/lfbf/a path=/select params={q=*:*&shards=[ff01::114]:33332/lfbf/a|http://127.0.0.1:50033/lfbf/a/collection1|[ff01::213]:33332/lfbf/a|[ff01::083]:33332/lfbf/a&sort=n_ti1+desc&wt=javabin&version=2} hits=68 status=0 QTime=44
[junit4] 2> 360779 INFO (qtp1699784955-4107) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/lfbf/a path=/select params={df=text&distrib=false&fl=id&shards.purpose=4&start=0&fsv=true&sort=n_ti1+desc&shard.url=[ff01::114]:33332/lfbf/a|http://127.0.0.1:50033/lfbf/a/collection1|[ff01::213]:33332/lfbf/a|[ff01::083]:33332/lfbf/a&rows=10&version=2&q=*:*&NOW=1502719950532&isShard=true&wt=javabin} hits=68 status=0 QTime=0
[junit4] 2> 360780 INFO (qtp1699784955-4107) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/lfbf/a path=/select params={q=*:*&df=text&distrib=false&shards.purpose=64&NOW=1502719950532&ids=11,144,3,15,125,17,117,129,108,10&isShard=true&shard.url=[ff01::114]:33332/lfbf/a|http://127.0.0.1:50033/lfbf/a/collection1|[ff01::213]:33332/lfbf/a|[ff01::083]:33332/lfbf/a&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 360781 INFO (qtp1699784955-4106) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/lfbf/a path=/select params={q=*:*&shards=[ff01::114]:33332/lfbf/a|http://127.0.0.1:50033/lfbf/a/collection1|[ff01::213]:33332/lfbf/a|[ff01::083]:33332/lfbf/a&sort=n_ti1+desc&wt=javabin&version=2} hits=68 status=0 QTime=2
[junit4] 2> 360782 INFO (qtp1699784955-4105) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/lfbf/a path=/select params={df=text&distrib=false&fl=id&shards.purpose=4&start=0&fsv=true&sort=n_ti1+desc&shard.url=[ff01::114]:33332/lfbf/a|http://127.0.0.1:50033/lfbf/a/collection1|[ff01::213]:33332/lfbf/a|[ff01::083]:33332/lfbf/a&rows=10&version=2&q=*:*&NOW=1502719950536&isShard=true&wt=javabin} hits=68 status=0 QTime=0
[junit4] 2> 360783 INFO (qtp1699784955-4105) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/lfbf/a path=/select params={q=*:*&df=text&distrib=false&shards.purpose=64&NOW=1502719950536&ids=11,144,3,15,125,17,117,129,108,10&isShard=true&shard.url=[ff01::114]:33332/lfbf/a|http://127.0.0.1:50033/lfbf/a/collection1|[ff01::213]:33332/lfbf/a|[ff01::083]:33332/lfbf/a&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 360784 INFO (qtp1699784955-4107) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/lfbf/a path=/select params={q=*:*&shards=[ff01::114]:33332/lfbf/a|http://127.0.0.1:50033/lfbf/a/collection1|[ff01::213]:33332/lfbf/a|[ff01::083]:33332/lfbf/a&sort=n_ti1+desc&wt=javabin&version=2} hits=68 status=0 QTime=2
[junit4] 2> 360785
[...truncated too long message...]
ethodMetric.java:144)
[junit4] 2> at org.apache.hadoop.metrics2.lib.MetricsRegistry.snapshot(MetricsRegistry.java:401)
[junit4] 2> at org.apache.hadoop.metrics2.lib.MetricsSourceBuilder$1.getMetrics(MetricsSourceBuilder.java:79)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMetrics(MetricsSourceAdapter.java:194)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.updateJmxCache(MetricsSourceAdapter.java:172)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.getMBeanInfo(MetricsSourceAdapter.java:151)
[junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getClassName(DefaultMBeanServerInterceptor.java:1804)
[junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.safeGetClassName(DefaultMBeanServerInterceptor.java:1595)
[junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanPermission(DefaultMBeanServerInterceptor.java:1813)
[junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:430)
[junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415)
[junit4] 2> at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546)
[junit4] 2> at org.apache.hadoop.metrics2.util.MBeans.unregister(MBeans.java:81)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stopMBeans(MetricsSourceAdapter.java:226)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.stop(MetricsSourceAdapter.java:211)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stopSources(MetricsSystemImpl.java:463)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.stop(MetricsSystemImpl.java:213)
[junit4] 2> at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.shutdown(MetricsSystemImpl.java:594)
[junit4] 2> at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdownInstance(DefaultMetricsSystem.java:72)
[junit4] 2> at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.shutdown(DefaultMetricsSystem.java:68)
[junit4] 2> at org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics.shutdown(NameNodeMetrics.java:171)
[junit4] 2> at org.apache.hadoop.hdfs.server.namenode.NameNode.stop(NameNode.java:872)
[junit4] 2> at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1726)
[junit4] 2> at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1705)
[junit4] 2> at org.apache.solr.cloud.hdfs.HdfsTestUtil.teardownClass(HdfsTestUtil.java:204)
[junit4] 2> at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.hdfsFailoverAfterClass(SharedFSAutoReplicaFailoverTest.java:97)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: java.lang.NullPointerException
[junit4] 2> at org.apache.hadoop.hdfs.server.blockmanagement.BlocksMap.size(BlocksMap.java:203)
[junit4] 2> at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.getTotalBlocks(BlockManager.java:3370)
[junit4] 2> at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlocksTotal(FSNamesystem.java:5729)
[junit4] 2> ... 54 more
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_97C3A76F18A3F955-001
[junit4] 2> Aug 14, 2017 3:05:14 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 34 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {id=PostingsFormat(name=Asserting), text=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128)))}, docValues:{_version_=DocValuesFormat(name=Memory)}, maxPointsInLeafNode=1792, maxMBSortInHeap=7.266282878290382, sim=RandomSimilarity(queryNorm=true): {}, locale=en-SG, timezone=EST5EDT
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=2,free=190067344,total=522190848
[junit4] 2> NOTE: All tests run in this JVM: [MoveReplicaHDFSTest, TestRestManager, SyncSliceTest, DefaultValueUpdateProcessorTest, JSONWriterTest, TestSolrConfigHandlerCloud, ConnectionManagerTest, DebugComponentTest, BadCopyFieldTest, AtomicUpdatesTest, TestHalfAndHalfDocValues, TestLeaderElectionWithEmptyReplica, TestPullReplicaErrorHandling, ConnectionReuseTest, DataDrivenBlockJoinTest, TestSolrCLIRunExample, TestComponentsName, CachingDirectoryFactoryTest, TestMergePolicyConfig, HighlighterConfigTest, UpdateParamsTest, SegmentsInfoRequestHandlerTest, DistributedFacetPivotLargeTest, TestStressLucene, TestPolicyCloud, ExternalFileFieldSortTest, TestDistributedSearch, MultiThreadedOCPTest, TestBadConfig, ShardSplitTest, TestManagedStopFilterFactory, DocumentAnalysisRequestHandlerTest, QueryResultKeyTest, V2ApiIntegrationTest, SolrMetricManagerTest, TestFieldCacheReopen, RAMDirectoryFactoryTest, TestRangeQuery, TestStressInPlaceUpdates, TestFieldSortValues, TestRequestStatusCollectionAPI, OverseerRolesTest, TestSubQueryTransformerDistrib, CollectionStateFormat2Test, TestLegacyTerms, TestLazyCores, RulesTest, PeerSyncTest, SimpleCollectionCreateDeleteTest, HdfsChaosMonkeyNothingIsSafeTest, TestClusterProperties, SolrIndexSplitterTest, CurrencyFieldTypeTest, AddBlockUpdateTest, TestStreamBody, TestLMDirichletSimilarityFactory, TestNumericTerms32, TestIndexingPerformance, HdfsNNFailoverTest, HighlighterMaxOffsetTest, HdfsTlogReplayBufferedWhileIndexingTest, EchoParamsTest, SuggesterWFSTTest, LeaderInitiatedRecoveryOnShardRestartTest, SolrXmlInZkTest, DistributedQueryComponentOptimizationTest, DistributedSpellCheckComponentTest, TestCollectionAPIs, FieldMutatingUpdateProcessorTest, TestCustomDocTransformer, TestTolerantUpdateProcessorCloud, TestSchemalessBufferedUpdates, CoreAdminRequestStatusTest, TestEmbeddedSolrServerConstructors, DeleteReplicaTest, TestSolrDeletionPolicy1, TestDistributedStatsComponentCardinality, TestLuceneMatchVersion, SortSpecParsingTest, TestPivotHelperCode, TestConfigSetsAPI, TestRandomDVFaceting, UUIDFieldTest, DistributedQueryElevationComponentTest, TestJsonFacetRefinement, TestDocumentBuilder, CleanupOldIndexTest, TestQuerySenderNoQuery, SolrGangliaReporterTest, TestSmileRequest, TestCloudPseudoReturnFields, SubstringBytesRefFilterTest, TolerantUpdateProcessorTest, TestFastOutputStream, PeerSyncReplicationTest, BlockDirectoryTest, TestFunctionQuery, TestDownShardTolerantSearch, TestWriterPerf, TestComplexPhraseLeadingWildcard, ReplaceNodeTest, JvmMetricsTest, TestWordDelimiterFilterFactory, SpellingQueryConverterTest, TestSolr4Spatial, TestRemoteStreaming, TestDynamicFieldCollectionResource, SearchHandlerTest, TestRandomFaceting, SolrSlf4jReporterTest, TestChildDocTransformer, TestIBSimilarityFactory, CursorPagingTest, ChaosMonkeyNothingIsSafeWithPullReplicasTest, TestConfig, CdcrRequestHandlerTest, CollectionsAPISolrJTest, DistributedQueueTest, DocValuesNotIndexedTest, ForceLeaderTest, MissingSegmentRecoveryTest, MoveReplicaTest, SaslZkACLProviderTest, SharedFSAutoReplicaFailoverTest]
[junit4] Completed [524/728 (5!)] on J0 in 100.78s, 1 test, 1 failure <<< FAILURES!
[...truncated 43978 lines...]
[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 28 - Still
unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/28/
3 tests failed.
FAILED: org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader
Error Message:
Doc with id=1 not found in http://127.0.0.1:53965/jqa/forceleader_test_collection due to: Path not found: /id; rsp={doc=null}
Stack Trace:
java.lang.AssertionError: Doc with id=1 not found in http://127.0.0.1:53965/jqa/forceleader_test_collection due to: Path not found: /id; rsp={doc=null}
at __randomizedtesting.SeedInfo.seed([A7A56C2D45AFBE43:413258ED7C2D4722]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.HttpPartitionTest.assertDocExists(HttpPartitionTest.java:603)
at org.apache.solr.cloud.HttpPartitionTest.assertDocsExistInAllReplicas(HttpPartitionTest.java:556)
at org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader(ForceLeaderTest.java:142)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_dv
Error Message:
java.lang.RuntimeException: Error from server at http://127.0.0.1:33958/solr/test_col_shard3_replica_n8: Failed synchronous update on shard StdNode: http://127.0.0.1:54589/solr/test_col_shard3_replica_n10/ update: org.apache.solr.client.solrj.request.UpdateRequest@46b4409e
Stack Trace:
java.util.concurrent.ExecutionException: java.lang.RuntimeException: Error from server at http://127.0.0.1:33958/solr/test_col_shard3_replica_n8: Failed synchronous update on shard StdNode: http://127.0.0.1:54589/solr/test_col_shard3_replica_n10/ update: org.apache.solr.client.solrj.request.UpdateRequest@46b4409e
at __randomizedtesting.SeedInfo.seed([A7A56C2D45AFBE43:91B10E6BCFF28452]:0)
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.checkField(TestStressCloudBlindAtomicUpdates.java:283)
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_dv(TestStressCloudBlindAtomicUpdates.java:195)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: Error from server at http://127.0.0.1:33958/solr/test_col_shard3_replica_n8: Failed synchronous update on shard StdNode: http://127.0.0.1:54589/solr/test_col_shard3_replica_n10/ update: org.apache.solr.client.solrj.request.UpdateRequest@46b4409e
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.run(TestStressCloudBlindAtomicUpdates.java:411)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
Caused by: org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from server at http://127.0.0.1:33958/solr/test_col_shard3_replica_n8: Failed synchronous update on shard StdNode: http://127.0.0.1:54589/solr/test_col_shard3_replica_n10/ update: org.apache.solr.client.solrj.request.UpdateRequest@46b4409e
at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:539)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:993)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:862)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:793)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195)
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.doRandomAtomicUpdate(TestStressCloudBlindAtomicUpdates.java:370)
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates$Worker.run(TestStressCloudBlindAtomicUpdates.java:399)
... 6 more
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:33958/solr/test_col_shard3_replica_n8: Failed synchronous update on shard StdNode: http://127.0.0.1:54589/solr/test_col_shard3_replica_n10/ update: org.apache.solr.client.solrj.request.UpdateRequest@46b4409e
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:627)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:516)
... 5 more
FAILED: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test
Error Message:
expected:<2> but was:<3>
Stack Trace:
java.lang.AssertionError: expected:<2> but was:<3>
at __randomizedtesting.SeedInfo.seed([A7A56C2D45AFBE43:2FF153F7EB53D3BB]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.assertSliceAndReplicaCount(SharedFSAutoReplicaFailoverTest.java:403)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.testBasics(SharedFSAutoReplicaFailoverTest.java:248)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test(SharedFSAutoReplicaFailoverTest.java:136)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 11488 lines...]
[junit4] Suite: org.apache.solr.cloud.ForceLeaderTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_A7A56C2D45AFBE43-001/init-core-data-001
[junit4] 2> 969192 WARN (SUITE-ForceLeaderTest-seed#[A7A56C2D45AFBE43]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=123 numCloses=123
[junit4] 2> 969192 INFO (SUITE-ForceLeaderTest-seed#[A7A56C2D45AFBE43]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 969193 INFO (SUITE-ForceLeaderTest-seed#[A7A56C2D45AFBE43]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
[junit4] 2> 969193 INFO (SUITE-ForceLeaderTest-seed#[A7A56C2D45AFBE43]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 969193 INFO (SUITE-ForceLeaderTest-seed#[A7A56C2D45AFBE43]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /jqa/
[junit4] 2> 969194 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 969195 INFO (Thread-1379) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 969195 INFO (Thread-1379) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 969196 ERROR (Thread-1379) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 969295 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.c.ZkTestServer start zk server on port:53016
[junit4] 2> 969315 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 969316 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 969317 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 969318 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 969318 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 969319 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 969320 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 969320 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 969321 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 969322 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 969322 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 969323 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use TLOG replicas unless explicitly asked otherwise
[junit4] 2> 969410 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 969411 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@39d11606{/jqa,null,AVAILABLE}
[junit4] 2> 969412 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@62cf1a66{HTTP/1.1,[http/1.1]}{127.0.0.1:51148}
[junit4] 2> 969412 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.e.j.s.Server Started @973115ms
[junit4] 2> 969412 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_A7A56C2D45AFBE43-001/tempDir-001/control/data, replicaType=NRT, hostContext=/jqa, hostPort=52522, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_A7A56C2D45AFBE43-001/control-001/cores}
[junit4] 2> 969412 ERROR (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 969412 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
[junit4] 2> 969412 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 969412 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 969412 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-13T12:52:05.274Z
[junit4] 2> 969425 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 969425 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_A7A56C2D45AFBE43-001/control-001/solr.xml
[junit4] 2> 969429 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@2d1eb290, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 969432 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53016/solr
[junit4] 2> 969517 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [n:127.0.0.1:52522_jqa ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 969518 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [n:127.0.0.1:52522_jqa ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:52522_jqa
[junit4] 2> 969518 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [n:127.0.0.1:52522_jqa ] o.a.s.c.Overseer Overseer (id=98476276125335556-127.0.0.1:52522_jqa-n_0000000000) starting
[junit4] 2> 969531 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [n:127.0.0.1:52522_jqa ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52522_jqa
[junit4] 2> 969532 INFO (zkCallback-13246-thread-1-processing-n:127.0.0.1:52522_jqa) [n:127.0.0.1:52522_jqa ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 969650 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [n:127.0.0.1:52522_jqa ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d1eb290
[junit4] 2> 969653 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [n:127.0.0.1:52522_jqa ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d1eb290
[junit4] 2> 969653 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [n:127.0.0.1:52522_jqa ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d1eb290
[junit4] 2> 969654 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [n:127.0.0.1:52522_jqa ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_A7A56C2D45AFBE43-001/control-001/cores
[junit4] 2> 969669 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 969670 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:53016/solr ready
[junit4] 2> 969670 INFO (SocketProxy-Acceptor-52522) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=57137,localport=52522], receiveBufferSize:531000
[junit4] 2> 969678 INFO (SocketProxy-Acceptor-52522) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=51148,localport=55050], receiveBufferSize=530904
[junit4] 2> 969679 INFO (qtp267414902-24098) [n:127.0.0.1:52522_jqa ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:52522_jqa&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 969681 INFO (OverseerThreadFactory-1667-thread-1-processing-n:127.0.0.1:52522_jqa) [n:127.0.0.1:52522_jqa ] o.a.s.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 969784 INFO (SocketProxy-Acceptor-52522) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=57139,localport=52522], receiveBufferSize:531000
[junit4] 2> 969785 INFO (SocketProxy-Acceptor-52522) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=51148,localport=55052], receiveBufferSize=530904
[junit4] 2> 969785 INFO (qtp267414902-24096) [n:127.0.0.1:52522_jqa ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 969786 INFO (qtp267414902-24096) [n:127.0.0.1:52522_jqa ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 969894 INFO (zkCallback-13246-thread-1-processing-n:127.0.0.1:52522_jqa) [n:127.0.0.1:52522_jqa ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 970798 INFO (qtp267414902-24096) [n:127.0.0.1:52522_jqa c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 970809 INFO (qtp267414902-24096) [n:127.0.0.1:52522_jqa c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
[junit4] 2> 970906 INFO (qtp267414902-24096) [n:127.0.0.1:52522_jqa c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 970921 INFO (qtp267414902-24096) [n:127.0.0.1:52522_jqa c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
[junit4] 2> 970923 INFO (qtp267414902-24096) [n:127.0.0.1:52522_jqa c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d1eb290
[junit4] 2> 970923 INFO (qtp267414902-24096) [n:127.0.0.1:52522_jqa c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 970923 INFO (qtp267414902-24096) [n:127.0.0.1:52522_jqa c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_A7A56C2D45AFBE43-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_A7A56C2D45AFBE43-001/control-001/cores/control_collection_shard1_replica_n1/data/]
[junit4] 2> 970931 INFO (qtp267414902-24096) [n:127.0.0.1:52522_jqa c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=2, maxMergeAtOnceExplicit=9, maxMergedSegmentMB=3.787109375, floorSegmentMB=0.771484375, forceMergeDeletesPctAllowed=26.067501615410904, segmentsPerTier=41.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 970933 WARN (qtp267414902-24096) [n:127.0.0.1:52522_jqa c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 970982 INFO (qtp267414902-24096) [n:127.0.0.1:52522_jqa c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 970982 INFO (qtp267414902-24096) [n:127.0.0.1:52522_jqa c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 970983 INFO (qtp267414902-24096) [n:127.0.0.1:52522_jqa c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 970983 INFO (qtp267414902-24096) [n:127.0.0.1:52522_jqa c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 970984 INFO (qtp267414902-24096) [n:127.0.0.1:52522_jqa c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=47, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 970984 INFO (qtp267414902-24096) [n:127.0.0.1:52522_jqa c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1656ae71[control_collection_shard1_replica_n1] main]
[junit4] 2> 970985 INFO (qtp267414902-24096) [n:127.0.0.1:52522_jqa c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 970985 INFO (qtp267414902-24096) [n:127.0.0.1:52522_jqa c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 970986 INFO (qtp267414902-24096) [n:127.0.0.1:52522_jqa c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 970987 INFO (searcherExecutor-1670-thread-1-processing-n:127.0.0.1:52522_jqa x:control_collection_shard1_replica_n1 s:shard1 c:control_collection) [n:127.0.0.1:52522_jqa c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@1656ae71[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 970987 INFO (qtp267414902-24096) [n:127.0.0.1:52522_jqa c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575620419884417024
[junit4] 2> 970993 INFO (qtp267414902-24096) [n:127.0.0.1:52522_jqa c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 970993 INFO (qtp267414902-24096) [n:127.0.0.1:52522_jqa c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 970993 INFO (qtp267414902-24096) [n:127.0.0.1:52522_jqa c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:52522/jqa/control_collection_shard1_replica_n1/
[junit4] 2> 970993 INFO (qtp267414902-24096) [n:127.0.0.1:52522_jqa c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 970993 INFO (qtp267414902-24096) [n:127.0.0.1:52522_jqa c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:52522/jqa/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 970993 INFO (qtp267414902-24096) [n:127.0.0.1:52522_jqa c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 970997 INFO (qtp267414902-24096) [n:127.0.0.1:52522_jqa c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:52522/jqa/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 971098 INFO (zkCallback-13246-thread-1-processing-n:127.0.0.1:52522_jqa) [n:127.0.0.1:52522_jqa ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 971147 INFO (qtp267414902-24096) [n:127.0.0.1:52522_jqa c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 971148 INFO (qtp267414902-24096) [n:127.0.0.1:52522_jqa c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1363
[junit4] 2> 971152 INFO (qtp267414902-24098) [n:127.0.0.1:52522_jqa ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 971253 INFO (zkCallback-13246-thread-1-processing-n:127.0.0.1:52522_jqa) [n:127.0.0.1:52522_jqa ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 971683 INFO (OverseerCollectionConfigSetProcessor-98476276125335556-127.0.0.1:52522_jqa-n_0000000000) [n:127.0.0.1:52522_jqa ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 972152 INFO (qtp267414902-24098) [n:127.0.0.1:52522_jqa ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:52522_jqa&wt=javabin&version=2} status=0 QTime=2472
[junit4] 2> 972163 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 972163 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:53016/solr ready
[junit4] 2> 972163 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 972164 INFO (SocketProxy-Acceptor-52522) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=57467,localport=52522], receiveBufferSize:531000
[junit4] 2> 972164 INFO (SocketProxy-Acceptor-52522) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=51148,localport=55380], receiveBufferSize=530904
[junit4] 2> 972173 INFO (qtp267414902-24095) [n:127.0.0.1:52522_jqa ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 972177 INFO (OverseerThreadFactory-1667-thread-2-processing-n:127.0.0.1:52522_jqa) [n:127.0.0.1:52522_jqa ] o.a.s.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 972178 WARN (OverseerThreadFactory-1667-thread-2-processing-n:127.0.0.1:52522_jqa) [n:127.0.0.1:52522_jqa ] o.a.s.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
[junit4] 2> 972382 INFO (qtp267414902-24095) [n:127.0.0.1:52522_jqa ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 972383 INFO (qtp267414902-24095) [n:127.0.0.1:52522_jqa ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=2&wt=javabin&version=2} status=0 QTime=209
[junit4] 2> 972478 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_A7A56C2D45AFBE43-001/shard-1-001 of type TLOG
[junit4] 2> 972479 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 972480 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2c126735{/jqa,null,AVAILABLE}
[junit4] 2> 972480 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@14c58a43{HTTP/1.1,[http/1.1]}{127.0.0.1:38419}
[junit4] 2> 972480 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.e.j.s.Server Started @976183ms
[junit4] 2> 972480 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_A7A56C2D45AFBE43-001/tempDir-001/jetty1, replicaType=TLOG, solrconfig=solrconfig.xml, hostContext=/jqa, hostPort=45016, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_A7A56C2D45AFBE43-001/shard-1-001/cores}
[junit4] 2> 972481 ERROR (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 972481 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
[junit4] 2> 972481 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 972481 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 972481 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-13T12:52:08.343Z
[junit4] 2> 972490 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 972490 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_A7A56C2D45AFBE43-001/shard-1-001/solr.xml
[junit4] 2> 972495 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@2d1eb290, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 972497 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53016/solr
[junit4] 2> 972503 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [n:127.0.0.1:45016_jqa ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 972504 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [n:127.0.0.1:45016_jqa ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 972506 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [n:127.0.0.1:45016_jqa ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45016_jqa
[junit4] 2> 972507 INFO (zkCallback-13246-thread-1-processing-n:127.0.0.1:52522_jqa) [n:127.0.0.1:52522_jqa ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 972507 INFO (zkCallback-13253-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 972523 INFO (zkCallback-13258-thread-1-processing-n:127.0.0.1:45016_jqa) [n:127.0.0.1:45016_jqa ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 972756 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [n:127.0.0.1:45016_jqa ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d1eb290
[junit4] 2> 972760 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [n:127.0.0.1:45016_jqa ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d1eb290
[junit4] 2> 972760 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [n:127.0.0.1:45016_jqa ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d1eb290
[junit4] 2> 972762 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [n:127.0.0.1:45016_jqa ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_A7A56C2D45AFBE43-001/shard-1-001/cores
[junit4] 2> 972794 INFO (qtp267414902-24100) [n:127.0.0.1:52522_jqa ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:45016_jqa&action=ADDREPLICA&collection=collection1&shard=shard2&type=TLOG&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 972802 INFO (OverseerCollectionConfigSetProcessor-98476276125335556-127.0.0.1:52522_jqa-n_0000000000) [n:127.0.0.1:52522_jqa ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 972803 INFO (OverseerThreadFactory-1667-thread-3-processing-n:127.0.0.1:52522_jqa) [n:127.0.0.1:52522_jqa ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:45016_jqa for creating new replica
[junit4] 2> 972805 INFO (SocketProxy-Acceptor-45016) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=53926,localport=45016], receiveBufferSize:531000
[junit4] 2> 972815 INFO (SocketProxy-Acceptor-45016) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=38419,localport=38262], receiveBufferSize=530904
[junit4] 2> 972816 INFO (qtp629808490-24146) [n:127.0.0.1:45016_jqa ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_t41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=TLOG
[junit4] 2> 972817 INFO (qtp629808490-24146) [n:127.0.0.1:45016_jqa ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 972920 INFO (zkCallback-13258-thread-1-processing-n:127.0.0.1:45016_jqa) [n:127.0.0.1:45016_jqa ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 973829 INFO (qtp629808490-24146) [n:127.0.0.1:45016_jqa c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 973839 INFO (qtp629808490-24146) [n:127.0.0.1:45016_jqa c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.s.IndexSchema [collection1_shard2_replica_t41] Schema name=test
[junit4] 2> 973924 INFO (qtp629808490-24146) [n:127.0.0.1:45016_jqa c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 973933 INFO (qtp629808490-24146) [n:127.0.0.1:45016_jqa c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_t41' using configuration from collection collection1, trusted=true
[junit4] 2> 973933 INFO (qtp629808490-24146) [n:127.0.0.1:45016_jqa c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_t41' (registry 'solr.core.collection1.shard2.replica_t41') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d1eb290
[junit4] 2> 973933 INFO (qtp629808490-24146) [n:127.0.0.1:45016_jqa c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 973933 INFO (qtp629808490-24146) [n:127.0.0.1:45016_jqa c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.SolrCore [[collection1_shard2_replica_t41] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_A7A56C2D45AFBE43-001/shard-1-001/cores/collection1_shard2_replica_t41], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_A7A56C2D45AFBE43-001/shard-1-001/cores/collection1_shard2_replica_t41/data/]
[junit4] 2> 973936 INFO (qtp629808490-24146) [n:127.0.0.1:45016_jqa c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=2, maxMergeAtOnceExplicit=9, maxMergedSegmentMB=3.787109375, floorSegmentMB=0.771484375, forceMergeDeletesPctAllowed=26.067501615410904, segmentsPerTier=41.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 973938 WARN (qtp629808490-24146) [n:127.0.0.1:45016_jqa c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 973968 INFO (qtp629808490-24146) [n:127.0.0.1:45016_jqa c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 973968 INFO (qtp629808490-24146) [n:127.0.0.1:45016_jqa c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 973969 INFO (qtp629808490-24146) [n:127.0.0.1:45016_jqa c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 973969 INFO (qtp629808490-24146) [n:127.0.0.1:45016_jqa c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 973970 INFO (qtp629808490-24146) [n:127.0.0.1:45016_jqa c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=47, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 973970 INFO (qtp629808490-24146) [n:127.0.0.1:45016_jqa c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.s.SolrIndexSearcher Opening [Searcher@3f09e920[collection1_shard2_replica_t41] main]
[junit4] 2> 973970 INFO (qtp629808490-24146) [n:127.0.0.1:45016_jqa c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 973971 INFO (qtp629808490-24146) [n:127.0.0.1:45016_jqa c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 973971 INFO (qtp629808490-24146) [n:127.0.0.1:45016_jqa c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 973972 INFO (searcherExecutor-1681-thread-1-processing-n:127.0.0.1:45016_jqa x:collection1_shard2_replica_t41 s:shard2 c:collection1) [n:127.0.0.1:45016_jqa c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.SolrCore [collection1_shard2_replica_t41] Registered new searcher Searcher@3f09e920[collection1_shard2_replica_t41] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 973972 INFO (qtp629808490-24146) [n:127.0.0.1:45016_jqa c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575620423014416384
[junit4] 2> 973977 INFO (qtp629808490-24146) [n:127.0.0.1:45016_jqa c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 973977 INFO (qtp629808490-24146) [n:127.0.0.1:45016_jqa c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 973977 INFO (qtp629808490-24146) [n:127.0.0.1:45016_jqa c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:45016/jqa/collection1_shard2_replica_t41/
[junit4] 2> 973977 INFO (qtp629808490-24146) [n:127.0.0.1:45016_jqa c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 973977 INFO (qtp629808490-24146) [n:127.0.0.1:45016_jqa c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.SyncStrategy http://127.0.0.1:45016/jqa/collection1_shard2_replica_t41/ has no replicas
[junit4] 2> 973977 INFO (qtp629808490-24146) [n:127.0.0.1:45016_jqa c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 973978 INFO (qtp629808490-24146) [n:127.0.0.1:45016_jqa c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.ZkController collection1_shard2_replica_t41 stopping background replication from leader
[junit4] 2> 973983 INFO (qtp629808490-24146) [n:127.0.0.1:45016_jqa c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:45016/jqa/collection1_shard2_replica_t41/ shard2
[junit4] 2> 974084 INFO (zkCallback-13258-thread-1-processing-n:127.0.0.1:45016_jqa) [n:127.0.0.1:45016_jqa ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 974134 INFO (qtp629808490-24146) [n:127.0.0.1:45016_jqa c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 974135 INFO (qtp629808490-24146) [n:127.0.0.1:45016_jqa c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_t41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=1318
[junit4] 2> 974138 INFO (qtp267414902-24100) [n:127.0.0.1:52522_jqa ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:45016_jqa&action=ADDREPLICA&collection=collection1&shard=shard2&type=TLOG&wt=javabin&version=2} status=0 QTime=1344
[junit4] 2> 974238 INFO (zkCallback-13258-thread-1-processing-n:127.0.0.1:45016_jqa) [n:127.0.0.1:45016_jqa ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 974240 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_A7A56C2D45AFBE43-001/shard-2-001 of type TLOG
[junit4] 2> 974240 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 974241 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4cff0ff{/jqa,null,AVAILABLE}
[junit4] 2> 974241 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@5e6cca17{HTTP/1.1,[http/1.1]}{127.0.0.1:41760}
[junit4] 2> 974241 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.e.j.s.Server Started @977944ms
[junit4] 2> 974241 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_A7A56C2D45AFBE43-001/tempDir-001/jetty2, replicaType=TLOG, solrconfig=solrconfig.xml, hostContext=/jqa, hostPort=36364, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_A7A56C2D45AFBE43-001/shard-2-001/cores}
[junit4] 2> 974242 ERROR (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 974243 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
[junit4] 2> 974243 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 974243 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 974243 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-13T12:52:10.105Z
[junit4] 2> 974245 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 974245 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_A7A56C2D45AFBE43-001/shard-2-001/solr.xml
[junit4] 2> 974249 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@2d1eb290, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 974251 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53016/solr
[junit4] 2> 974259 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [n:127.0.0.1:36364_jqa ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 974260 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [n:127.0.0.1:36364_jqa ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 974263 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [n:127.0.0.1:36364_jqa ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36364_jqa
[junit4] 2> 974264 INFO (zkCallback-13246-thread-1-processing-n:127.0.0.1:52522_jqa) [n:127.0.0.1:52522_jqa ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 974264 INFO (zkCallback-13258-thread-1-processing-n:127.0.0.1:45016_jqa) [n:127.0.0.1:45016_jqa ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 974265 INFO (zkCallback-13253-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 974266 INFO (zkCallback-13264-thread-1-processing-n:127.0.0.1:36364_jqa) [n:127.0.0.1:36364_jqa ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 974326 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [n:127.0.0.1:36364_jqa ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d1eb290
[junit4] 2> 974355 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [n:127.0.0.1:36364_jqa ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d1eb290
[junit4] 2> 974355 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [n:127.0.0.1:36364_jqa ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d1eb290
[junit4] 2> 974357 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [n:127.0.0.1:36364_jqa ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_A7A56C2D45AFBE43-001/shard-2-001/cores
[junit4] 2> 974378 INFO (qtp267414902-24096) [n:127.0.0.1:52522_jqa ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:36364_jqa&action=ADDREPLICA&collection=collection1&shard=shard1&type=TLOG&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 974386 INFO (OverseerCollectionConfigSetProcessor-98476276125335556-127.0.0.1:52522_jqa-n_0000000000) [n:127.0.0.1:52522_jqa ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 974387 INFO (OverseerThreadFactory-1667-thread-4-processing-n:127.0.0.1:52522_jqa) [n:127.0.0.1:52522_jqa ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:36364_jqa for creating new replica
[junit4] 2> 974388 INFO (SocketProxy-Acceptor-36364) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=53548,localport=36364], receiveBufferSize:531000
[junit4] 2> 974393 INFO (SocketProxy-Acceptor-36364) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=41760,localport=58746], receiveBufferSize=530904
[junit4] 2> 974394 INFO (qtp504797985-24179) [n:127.0.0.1:36364_jqa ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_t43&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=TLOG
[junit4] 2> 974394 INFO (qtp504797985-24179) [n:127.0.0.1:36364_jqa ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 974498 INFO (zkCallback-13258-thread-1-processing-n:127.0.0.1:45016_jqa) [n:127.0.0.1:45016_jqa ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 974498 INFO (zkCallback-13264-thread-1-processing-n:127.0.0.1:36364_jqa) [n:127.0.0.1:36364_jqa ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 975407 INFO (qtp504797985-24179) [n:127.0.0.1:36364_jqa c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 975417 INFO (qtp504797985-24179) [n:127.0.0.1:36364_jqa c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.s.IndexSchema [collection1_shard1_replica_t43] Schema name=test
[junit4] 2> 975510 INFO (qtp504797985-24179) [n:127.0.0.1:36364_jqa c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 975529 INFO (qtp504797985-24179) [n:127.0.0.1:36364_jqa c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_t43' using configuration from collection collection1, trusted=true
[junit4] 2> 975530 INFO (qtp504797985-24179) [n:127.0.0.1:36364_jqa c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_t43' (registry 'solr.core.collection1.shard1.replica_t43') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d1eb290
[junit4] 2> 975530 INFO (qtp504797985-24179) [n:127.0.0.1:36364_jqa c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 975530 INFO (qtp504797985-24179) [n:127.0.0.1:36364_jqa c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.SolrCore [[collection1_shard1_replica_t43] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_A7A56C2D45AFBE43-001/shard-2-001/cores/collection1_shard1_replica_t43], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_A7A56C2D45AFBE43-001/shard-2-001/cores/collection1_shard1_replica_t43/data/]
[junit4] 2> 975532 INFO (qtp504797985-24179) [n:127.0.0.1:36364_jqa c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=2, maxMergeAtOnceExplicit=9, maxMergedSegmentMB=3.787109375, floorSegmentMB=0.771484375, forceMergeDeletesPctAllowed=26.067501615410904, segmentsPerTier=41.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 975535 WARN (qtp504797985-24179) [n:127.0.0.1:36364_jqa c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 975567 INFO (qtp504797985-24179) [n:127.0.0.1:36364_jqa c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 975567 INFO (qtp504797985-24179) [n:127.0.0.1:36364_jqa c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 975568 INFO (qtp504797985-24179) [n:127.0.0.1:36364_jqa c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 975568 INFO (qtp504797985-24179) [n:127.0.0.1:36364_jqa c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 975569 INFO (qtp504797985-24179) [n:127.0.0.1:36364_jqa c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=47, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 975569 INFO (qtp504797985-24179) [n:127.0.0.1:36364_jqa c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.s.SolrIndexSearcher Opening [Searcher@37ec5845[collection1_shard1_replica_t43] main]
[junit4] 2> 975570 INFO (qtp504797985-24179) [n:127.0.0.1:36364_jqa c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 975571 INFO (qtp504797985-24179) [n:127.0.0.1:36364_jqa c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 975571 INFO (qtp504797985-24179) [n:127.0.0.1:36364_jqa c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 975572 INFO (searcherExecutor-1692-thread-1-processing-n:127.0.0.1:36364_jqa x:collection1_shard1_replica_t43 s:shard1 c:collection1) [n:127.0.0.1:36364_jqa c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.SolrCore [collection1_shard1_replica_t43] Registered new searcher Searcher@37ec5845[collection1_shard1_replica_t43] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 975572 INFO (qtp504797985-24179) [n:127.0.0.1:36364_jqa c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575620424692137984
[junit4] 2> 975577 INFO (qtp504797985-24179) [n:127.0.0.1:36364_jqa c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 975577 INFO (qtp504797985-24179) [n:127.0.0.1:36364_jqa c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 975577 INFO (qtp504797985-24179) [n:127.0.0.1:36364_jqa c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:36364/jqa/collection1_shard1_replica_t43/
[junit4] 2> 975577 INFO (qtp504797985-24179) [n:127.0.0.1:36364_jqa c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 975577 INFO (qtp504797985-24179) [n:127.0.0.1:36364_jqa c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.SyncStrategy http://127.0.0.1:36364/jqa/collection1_shard1_replica_t43/ has no replicas
[junit4] 2> 975577 INFO (qtp504797985-24179) [n:127.0.0.1:36364_jqa c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 975577 INFO (qtp504797985-24179) [n:127.0.0.1:36364_jqa c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.ZkController collection1_shard1_replica_t43 stopping background replication from leader
[junit4] 2> 975579 INFO (qtp504797985-24179) [n:127.0.0.1:36364_jqa c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:36364/jqa/collection1_shard1_replica_t43/ shard1
[junit4] 2> 975687 INFO (zkCallback-13264-thread-1-processing-n:127.0.0.1:36364_jqa) [n:127.0.0.1:36364_jqa ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 975687 INFO (zkCallback-13258-thread-1-processing-n:127.0.0.1:45016_jqa) [n:127.0.0.1:45016_jqa ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 975736 INFO (qtp504797985-24179) [n:127.0.0.1:36364_jqa c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 975737 INFO (qtp504797985-24179) [n:127.0.0.1:36364_jqa c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_t43&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=1342
[junit4] 2> 975745 INFO (qtp267414902-24096) [n:127.0.0.1:52522_jqa ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:36364_jqa&action=ADDREPLICA&collection=collection1&shard=shard1&type=TLOG&wt=javabin&version=2} status=0 QTime=1366
[junit4] 2> 975846 INFO (zkCallback-13264-thread-1-processing-n:127.0.0.1:36364_jqa) [n:127.0.0.1:36364_jqa ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 975846 INFO (zkCallback-13258-thread-1-processing-n:127.0.0.1:45016_jqa) [n:127.0.0.1:45016_jqa ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 975853 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_A7A56C2D45AFBE43-001/shard-3-001 of type TLOG
[junit4] 2> 975854 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 975855 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2d008146{/jqa,null,AVAILABLE}
[junit4] 2> 975855 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@335dba2a{HTTP/1.1,[http/1.1]}{127.0.0.1:57903}
[junit4] 2> 975855 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.e.j.s.Server Started @979558ms
[junit4] 2> 975855 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_A7A56C2D45AFBE43-001/tempDir-001/jetty3, replicaType=TLOG, solrconfig=solrconfig.xml, hostContext=/jqa, hostPort=54128, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_A7A56C2D45AFBE43-001/shard-3-001/cores}
[junit4] 2> 975856 ERROR (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 975857 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
[junit4] 2> 975865 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 975865 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 975865 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-13T12:52:11.727Z
[junit4] 2> 975867 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 975867 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_A7A56C2D45AFBE43-001/shard-3-001/solr.xml
[junit4] 2> 975872 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@2d1eb290, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 975874 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53016/solr
[junit4] 2> 975880 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [n:127.0.0.1:54128_jqa ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 975881 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [n:127.0.0.1:54128_jqa ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 975882 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [n:127.0.0.1:54128_jqa ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54128_jqa
[junit4] 2> 975883 INFO (zkCallback-13258-thread-1-processing-n:127.0.0.1:45016_jqa) [n:127.0.0.1:45016_jqa ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 975883 INFO (zkCallback-13264-thread-1-processing-n:127.0.0.1:36364_jqa) [n:127.0.0.1:36364_jqa ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 975883 INFO (zkCallback-13253-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 975883 INFO (zkCallback-13246-thread-2-processing-n:127.0.0.1:52522_jqa) [n:127.0.0.1:52522_jqa ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 975886 INFO (zkCallback-13270-thread-1-processing-n:127.0.0.1:54128_jqa) [n:127.0.0.1:54128_jqa ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 976027 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [n:127.0.0.1:54128_jqa ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d1eb290
[junit4] 2> 976031 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [n:127.0.0.1:54128_jqa ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d1eb290
[junit4] 2> 976031 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [n:127.0.0.1:54128_jqa ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d1eb290
[junit4] 2> 976032 INFO (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[A7A56C2D45AFBE43]) [n:127.0.0.1:54128_jqa ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_A7A56C2D45AFBE43-001/shard-3-001/cores
[junit4] 2> 976061 INFO (qtp267414902-24098) [n:127.0.0.1:52522_jqa ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:54128_jqa&action=ADDREPLICA&collection=collection1&shard=shard2&type=TLOG&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 976063 INFO (OverseerCollectionConfigSetProcessor-98476276125335556-127.0.0.1:52522_jqa-n_0000000000) [n:127.0.0.1:52522_jqa ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000006 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 976064 INFO (OverseerThreadFactory-1667-thread-5-processing-n:127.0.0.1:52522_jqa) [n:127.0.0.1:52522_jqa ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:54128_jqa for creating new replica
[junit4] 2> 976065 INFO (SocketProxy-Acceptor-54128) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=43971,localport=54128], receiveBufferSize:531000
[junit4] 2> 976065 INFO (SocketProxy-Acceptor-54128) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=57903,localport=43429], receiveBufferSize=530904
[junit4] 2> 976071 INFO (qtp1778689948-24211) [n:127.0.0.1:54128_jqa ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_t45&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=TLOG
[junit4] 2> 976072 INFO (qtp1778689948-24211) [n:127.0.0.1:54128_jqa ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 976175 INFO (zkCallback-13270-thread-1-processing-n:127.0.0.1:54128_jqa) [n:127.0.0.1:54128_jqa ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 976175 INFO (zkCallback-13264-thread-1-processing-n:127.0.0.1:36364_jqa) [n:127.0.0.1:36364_jqa ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 976175 INFO (zkCallback-13258-thread-1-processing-n:127.0.0.1:45016_jqa) [n:127.0.0.1:45016_jqa ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 977085 INFO (qtp1778689948-24211) [n:127.0.0.1:54128_jqa c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 977096 INFO (qtp1778689948-24211) [n:127.0.0.1:54128_jqa c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.s.IndexSchema [collection1_shard2_replica_t45] Schema name=test
[junit4] 2> 977187 INFO (qtp1778689948-24211) [n:127.0.0.1:54128_jqa c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 977202 INFO (qtp1778689948-24211) [n:127.0.0.1:54128_jqa c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_t45' using configuration from collection collection1, trusted=true
[junit4] 2> 977202 INFO (qtp1778689948-24211) [n:127.0.0.1:54128_jqa c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_t45' (registry 'solr.core.collection1.shard2.replica_t45') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2d1eb290
[junit4] 2> 977202 INFO (qtp1778689948-24211) [n:127.0.0.1:54128_jqa c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 977202 INFO (qtp1778689948-24211) [n:127.0.0.1:54128_jqa c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.c.SolrCore [[collection1_shard2_replica_t45] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_A7A56C2D45AFBE43-001/shard-3-001/cores/collection1_shard2_replica_t45], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_A7A56C2D45AFBE43-001/shard-3-001/cores/collection1_s
[...truncated too long message...]
0.0.1:32781_solr
[junit4] 2> 3939715 INFO (coreCloseExecutor-6012-thread-1) [n:127.0.0.1:32781_solr c:test_col s:shard1 r:core_node3 x:test_col_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.test_col.shard1.replica_n1, tag=1584051699
[junit4] 2> 3939718 INFO (coreCloseExecutor-6012-thread-1) [n:127.0.0.1:32781_solr c:test_col s:shard1 r:core_node3 x:test_col_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.test_col.shard1.leader, tag=1584051699
[junit4] 2> 3941150 WARN (zkCallback-2205-thread-6-processing-n:127.0.0.1:54127_solr) [n:127.0.0.1:54127_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 3941151 INFO (jetty-closer-2154-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@526011de{/solr,null,UNAVAILABLE}
[junit4] 2> 3941157 WARN (zkCallback-2202-thread-6-processing-n:127.0.0.1:41051_solr) [n:127.0.0.1:41051_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 3941157 WARN (zkCallback-2201-thread-7-processing-n:127.0.0.1:37127_solr) [n:127.0.0.1:37127_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 3941157 INFO (jetty-closer-2154-thread-3) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@bad21c4{/solr,null,UNAVAILABLE}
[junit4] 2> 3941157 INFO (jetty-closer-2154-thread-8) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@19cddfac{/solr,null,UNAVAILABLE}
[junit4] 2> 3941162 WARN (zkCallback-2207-thread-6-processing-n:127.0.0.1:54589_solr) [n:127.0.0.1:54589_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 3941162 INFO (jetty-closer-2154-thread-10) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@79e8c333{/solr,null,UNAVAILABLE}
[junit4] 2> 3941162 WARN (zkCallback-2197-thread-6-processing-n:127.0.0.1:45583_solr) [n:127.0.0.1:45583_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 3941162 INFO (jetty-closer-2154-thread-4) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5ce1b0e3{/solr,null,UNAVAILABLE}
[junit4] 2> 3941210 WARN (zkCallback-2193-thread-8-processing-n:127.0.0.1:44017_solr) [n:127.0.0.1:44017_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 3941210 INFO (jetty-closer-2154-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@4cb59ad1{/solr,null,UNAVAILABLE}
[junit4] 2> 3941219 WARN (zkCallback-2203-thread-5-processing-n:127.0.0.1:32781_solr) [n:127.0.0.1:32781_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 3941220 INFO (jetty-closer-2154-thread-6) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@4220cb79{/solr,null,UNAVAILABLE}
[junit4] 2> 3941221 ERROR (SUITE-TestStressCloudBlindAtomicUpdates-seed#[A7A56C2D45AFBE43]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 3941222 INFO (SUITE-TestStressCloudBlindAtomicUpdates-seed#[A7A56C2D45AFBE43]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:39073 39073
[junit4] 2> 3941325 INFO (Thread-207762) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:39073 39073
[junit4] 2> 3941330 WARN (Thread-207762) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 11 /solr/aliases.json
[junit4] 2> 11 /solr/clusterprops.json
[junit4] 2> 10 /solr/configs/org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates_config-set
[junit4] 2> 10 /solr/security.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 1892 /solr/collections/test_col/state.json
[junit4] 2> 11 /solr/clusterstate.json
[junit4] 2> 3 /solr/overseer_elect/election/98476450076819477-127.0.0.1:44017_solr-n_0000000000
[junit4] 2> 2 /solr/overseer_elect/election/98476450076819480-127.0.0.1:45583_solr-n_0000000005
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 11 /solr/live_nodes
[junit4] 2> 11 /solr/collections
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.TestStressCloudBlindAtomicUpdates_A7A56C2D45AFBE43-001
[junit4] 2> Aug 13, 2017 1:41:36 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=DummyCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=DUMMY, chunkSize=5, maxDocsPerChunk=392, blockSize=3), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=DUMMY, chunkSize=5, blockSize=3)), sim=RandomSimilarity(queryNorm=true): {}, locale=ar-OM, timezone=Australia/Canberra
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=40451128,total=533200896
[junit4] 2> NOTE: All tests run in this JVM: [CustomCollectionTest, QueryEqualityTest, TestCloudSearcherWarming, TestLMJelinekMercerSimilarityFactory, TestDistributedMissingSort, ChangedSchemaMergeTest, HdfsBasicDistributedZkTest, PreAnalyzedFieldManagedSchemaCloudTest, TestDistribIDF, TestBinaryResponseWriter, CoreSorterTest, MultiTermTest, AliasIntegrationTest, TermsComponentTest, DateMathParserTest, XsltUpdateRequestHandlerTest, SolrCloudExampleTest, UpdateLogTest, SolrMetricsIntegrationTest, SpellCheckCollatorWithCollapseTest, NumericFieldsTest, TestNamedUpdateProcessors, RuleEngineTest, TestFieldTypeResource, TestSchemaSimilarityResource, TestReversedWildcardFilterFactory, TestUniqueKeyFieldResource, DistributedFacetExistsSmallTest, TestNestedDocsSort, AsyncCallRequestStatusResponseTest, OutOfBoxZkACLAndCredentialsProvidersTest, TestTrie, SecurityConfHandlerTest, TestCustomSort, TestRuleBasedAuthorizationPlugin, CollectionReloadTest, SolrTestCaseJ4Test, TestGroupingSearch, HdfsDirectoryFactoryTest, ConfigSetsAPITest, TestZkChroot, PrimitiveFieldTypeTest, TestRTimerTree, SimplePostToolTest, DeleteStatusTest, TestRecovery, TestSchemaManager, TestXmlQParserPlugin, RemoteQueryErrorTest, TestPerFieldSimilarityWithDefaultOverride, CdcrUpdateLogTest, NotRequiredUniqueKeyTest, TestBulkSchemaConcurrent, ShowFileRequestHandlerTest, StatsComponentTest, TestGraphTermsQParserPlugin, TestSolrQueryResponse, LeaderElectionContextKeyTest, PathHierarchyTokenizerFactoryTest, TestMiniSolrCloudClusterSSL, SortByFunctionTest, BlockJoinFacetSimpleTest, HdfsDirectoryTest, TestFastWriter, TestManagedResourceStorage, PreAnalyzedFieldTest, SolrJmxReporterTest, SparseHLLTest, TlogReplayBufferedWhileIndexingTest, DocValuesNotIndexedTest, UtilsToolTest, RequiredFieldsTest, CopyFieldTest, TestSubQueryTransformerCrossCore, TestJmxIntegration, TestRandomRequestDistribution, TestJsonRequest, SpellPossibilityIteratorTest, TestRandomCollapseQParserPlugin, TestCollationFieldDocValues, TestLeaderInitiatedRecoveryThread, TestManagedSchemaAPI, TestMinMaxOnMultiValuedField, BJQParserTest, TestSolrCloudWithDelegationTokens, FacetPivotSmallTest, DistanceUnitsTest, SuggestComponentTest, TestUtils, InfoHandlerTest, TestDistribDocBasedVersion, TestMacros, BigEndianAscendingWordSerializerTest, CursorMarkTest, UniqFieldsUpdateProcessorFactoryTest, TestStressUserVersions, RestartWhileUpdatingTest, DistributedFacetPivotSmallAdvancedTest, BlockCacheTest, SignatureUpdateProcessorFactoryTest, TestRequestForwarding, DirectUpdateHandlerOptimizeTest, TestCodecSupport, RegexBoostProcessorTest, SuggesterFSTTest, TestCloudDeleteByQuery, SolrCoreCheckLockOnStartupTest, TestConfigSetsAPIZkFailure, TestNumericRangeQuery32, TestSort, DeleteNodeTest, HdfsCollectionsAPIDistributedZkTest, TestSurroundQueryParser, DocumentBuilderTest, TestHdfsUpdateLog, TestLegacyNumericUtils, TestInitParams, ShardRoutingCustomTest, TestDistributedSearch, TestDFRSimilarityFactory, TestBM25SimilarityFactory, URLClassifyProcessorTest, TestSuggestSpellingConverter, DateFieldTest, ClusterStateTest, SliceStateTest, SystemInfoHandlerTest, CircularListTest, CursorPagingTest, TestCursorMarkWithoutUniqueKey, TestHighlightDedupGrouping, TestSimpleTrackingShardHandler, TestTolerantSearch, TestEmbeddedSolrServerConstructors, TestJettySolrRunner, ConnectionReuseTest, AssignTest, CdcrReplicationDistributedZkTest, CdcrVersionReplicationTest, CleanupOldIndexTest, CollectionStateFormat2Test, CollectionsAPIAsyncDistributedZkTest, DeleteInactiveReplicaTest, DeleteShardTest, DistribDocExpirationUpdateProcessorTest, LeaderInitiatedRecoveryOnCommitTest, OverriddenZkACLAndCredentialsProvidersTest, OverseerModifyCollectionTest, PeerSyncReplicationTest, SharedFSAutoReplicaFailoverTest, SharedFSAutoReplicaFailoverUtilsTest, TestCloudJSONFacetJoinDomain, TestRequestStatusCollectionAPI, TestSizeLimitedDistributedMap, TestStressCloudBlindAtomicUpdates]
[junit4] Completed [574/728 (3!)] on J2 in 317.80s, 5 tests, 1 error <<< FAILURES!
[...truncated 43794 lines...]
[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 27 - Failure
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/27/
11 tests failed.
FAILED: org.apache.lucene.search.TestInetAddressRangeQueries.testRandomBig
Error Message:
GC overhead limit exceeded
Stack Trace:
java.lang.OutOfMemoryError: GC overhead limit exceeded
at __randomizedtesting.SeedInfo.seed([4B58E77EC66368F0:CC0F9AF1573A1470]:0)
at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectField$IntArrayWriter.get(DirectPostingsFormat.java:295)
at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectField.<init>(DirectPostingsFormat.java:413)
at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectFields.<init>(DirectPostingsFormat.java:129)
at org.apache.lucene.codecs.memory.DirectPostingsFormat.fieldsProducer(DirectPostingsFormat.java:113)
at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:292)
at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:372)
at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:112)
at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:78)
at org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:702)
at org.apache.lucene.util.TestUtil.checkIndex(TestUtil.java:300)
at org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:865)
at org.apache.lucene.util.IOUtils.close(IOUtils.java:89)
at org.apache.lucene.util.IOUtils.close(IOUtils.java:76)
at org.apache.lucene.search.BaseRangeFieldQueryTestCase.verify(BaseRangeFieldQueryTestCase.java:293)
at org.apache.lucene.search.BaseRangeFieldQueryTestCase.doTestRandom(BaseRangeFieldQueryTestCase.java:160)
at org.apache.lucene.search.BaseRangeFieldQueryTestCase.testRandomBig(BaseRangeFieldQueryTestCase.java:75)
at org.apache.lucene.search.TestInetAddressRangeQueries.testRandomBig(TestInetAddressRangeQueries.java:81)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
FAILED: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.test
Error Message:
The Monkey ran for over 45 seconds and no jetties were stopped - this is worth investigating!
Stack Trace:
java.lang.AssertionError: The Monkey ran for over 45 seconds and no jetties were stopped - this is worth investigating!
at __randomizedtesting.SeedInfo.seed([8DBA7CAC21166B83:5EE43768FEA067B]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.ChaosMonkey.stopTheMonkey(ChaosMonkey.java:587)
at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.test(ChaosMonkeySafeLeaderTest.java:133)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test
Error Message:
GC overhead limit exceeded
Stack Trace:
java.lang.OutOfMemoryError: GC overhead limit exceeded
FAILED: org.apache.solr.cloud.HttpPartitionTest.test
Error Message:
Doc with id=934 not found in http://127.0.0.1:55511/c8n_1x2 due to: Path not found: /id; rsp={doc=null}
Stack Trace:
java.lang.AssertionError: Doc with id=934 not found in http://127.0.0.1:55511/c8n_1x2 due to: Path not found: /id; rsp={doc=null}
at __randomizedtesting.SeedInfo.seed([8DBA7CAC21166B83:5EE43768FEA067B]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.HttpPartitionTest.assertDocExists(HttpPartitionTest.java:603)
at org.apache.solr.cloud.HttpPartitionTest.assertDocsExistInAllReplicas(HttpPartitionTest.java:558)
at org.apache.solr.cloud.HttpPartitionTest.testRf2(HttpPartitionTest.java:379)
at org.apache.solr.cloud.HttpPartitionTest.test(HttpPartitionTest.java:132)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest.test
Error Message:
The Monkey ran for over 45 seconds and no jetties were stopped - this is worth investigating!
Stack Trace:
java.lang.AssertionError: The Monkey ran for over 45 seconds and no jetties were stopped - this is worth investigating!
at __randomizedtesting.SeedInfo.seed([8DBA7CAC21166B83:5EE43768FEA067B]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.ChaosMonkey.stopTheMonkey(ChaosMonkey.java:587)
at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.test(ChaosMonkeySafeLeaderTest.java:133)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest
Error Message:
2 threads leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest: 1) Thread[id=36221, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-HdfsChaosMonkeySafeLeaderTest] at java.lang.Thread.sleep(Native Method) at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135) 2) Thread[id=36222, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-HdfsChaosMonkeySafeLeaderTest] at java.lang.Thread.sleep(Native Method) at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest:
1) Thread[id=36221, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-HdfsChaosMonkeySafeLeaderTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135)
2) Thread[id=36222, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-HdfsChaosMonkeySafeLeaderTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135)
at __randomizedtesting.SeedInfo.seed([8DBA7CAC21166B83]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=36221, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-HdfsChaosMonkeySafeLeaderTest] at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:328) at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228) at org.apache.solr.common.util.JavaBinCodec.writeSolrInputDocument(JavaBinCodec.java:616) at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:355) at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228) at org.apache.solr.common.util.JavaBinCodec.writeMapEntry(JavaBinCodec.java:764) at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:383) at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228) at org.apache.solr.common.util.JavaBinCodec.writeIterator(JavaBinCodec.java:705) at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:367) at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228) at org.apache.solr.common.util.JavaBinCodec.writeNamedList(JavaBinCodec.java:223) at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:330) at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228) at org.apache.solr.common.util.JavaBinCodec.marshal(JavaBinCodec.java:155) at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.marshal(JavaBinUpdateRequestCodec.java:91) at org.apache.solr.client.solrj.impl.BinaryRequestWriter.getContentStream(BinaryRequestWriter.java:67) at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:94) at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:104) at org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:382) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195) at org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:176) at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:115) 2) Thread[id=36222, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-HdfsChaosMonkeySafeLeaderTest] at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:328) at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228) at org.apache.solr.common.util.JavaBinCodec.writeMapEntry(JavaBinCodec.java:764) at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:383) at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228) at org.apache.solr.common.util.JavaBinCodec.writeIterator(JavaBinCodec.java:705) at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:367) at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228) at org.apache.solr.common.util.JavaBinCodec.writeNamedList(JavaBinCodec.java:223) at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:330) at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228) at org.apache.solr.common.util.JavaBinCodec.marshal(JavaBinCodec.java:155) at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.marshal(JavaBinUpdateRequestCodec.java:91) at org.apache.solr.client.solrj.impl.BinaryRequestWriter.getContentStream(BinaryRequestWriter.java:67) at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:94) at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:104) at org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:382) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195) at org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:176) at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:115)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=36221, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-HdfsChaosMonkeySafeLeaderTest]
at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:328)
at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
at org.apache.solr.common.util.JavaBinCodec.writeSolrInputDocument(JavaBinCodec.java:616)
at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:355)
at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
at org.apache.solr.common.util.JavaBinCodec.writeMapEntry(JavaBinCodec.java:764)
at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:383)
at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
at org.apache.solr.common.util.JavaBinCodec.writeIterator(JavaBinCodec.java:705)
at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:367)
at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
at org.apache.solr.common.util.JavaBinCodec.writeNamedList(JavaBinCodec.java:223)
at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:330)
at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
at org.apache.solr.common.util.JavaBinCodec.marshal(JavaBinCodec.java:155)
at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.marshal(JavaBinUpdateRequestCodec.java:91)
at org.apache.solr.client.solrj.impl.BinaryRequestWriter.getContentStream(BinaryRequestWriter.java:67)
at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:94)
at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:104)
at org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:382)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195)
at org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:176)
at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:115)
2) Thread[id=36222, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-HdfsChaosMonkeySafeLeaderTest]
at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:328)
at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
at org.apache.solr.common.util.JavaBinCodec.writeMapEntry(JavaBinCodec.java:764)
at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:383)
at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
at org.apache.solr.common.util.JavaBinCodec.writeIterator(JavaBinCodec.java:705)
at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:367)
at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
at org.apache.solr.common.util.JavaBinCodec.writeNamedList(JavaBinCodec.java:223)
at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:330)
at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
at org.apache.solr.common.util.JavaBinCodec.marshal(JavaBinCodec.java:155)
at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.marshal(JavaBinUpdateRequestCodec.java:91)
at org.apache.solr.client.solrj.impl.BinaryRequestWriter.getContentStream(BinaryRequestWriter.java:67)
at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:94)
at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:104)
at org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:382)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195)
at org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:176)
at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:115)
at __randomizedtesting.SeedInfo.seed([8DBA7CAC21166B83]:0)
FAILED: org.apache.solr.core.TestJmxIntegration.testJmxRegistration
Error Message:
getMBeanInfo threw RuntimeException
Stack Trace:
javax.management.RuntimeMBeanException: getMBeanInfo threw RuntimeException
at __randomizedtesting.SeedInfo.seed([8DBA7CAC21166B83:36B18964C5733E6]:0)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBeanInfo(DefaultMBeanServerInterceptor.java:1384)
at com.sun.jmx.mbeanserver.JmxMBeanServer.getMBeanInfo(JmxMBeanServer.java:920)
at org.apache.solr.core.TestJmxIntegration.testJmxRegistration(TestJmxIntegration.java:108)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.lucene.store.AlreadyClosedException: Already closed
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:337)
at org.apache.solr.handler.ReplicationHandler.loadReplicationProperties(ReplicationHandler.java:1131)
at org.apache.solr.handler.ReplicationHandler.lambda$initializeMetrics$8(ReplicationHandler.java:895)
at org.apache.solr.metrics.MetricsMap.getValue(MetricsMap.java:75)
at org.apache.solr.metrics.MetricsMap.getMBeanInfo(MetricsMap.java:143)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBeanInfo(DefaultMBeanServerInterceptor.java:1378)
... 41 more
FAILED: org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload
Error Message:
expected:<[{indexVersion=1502535335192,generation=2,filelist=[_97.fdt, _97.fdx, _97.fnm, _97.nvd, _97.nvm, _97.si, _97_Lucene50_0.doc, _97_Lucene50_0.tim, _97_Lucene50_0.tip, _99.fdt, _99.fdx, _99.fnm, _99.nvd, _99.nvm, _99.si, _99_Lucene50_0.doc, _99_Lucene50_0.tim, _99_Lucene50_0.tip, _9c.fdt, _9c.fdx, _9c.fnm, _9c.nvd, _9c.nvm, _9c.si, _9c_Lucene50_0.doc, _9c_Lucene50_0.tim, _9c_Lucene50_0.tip, _9l.fdt, _9l.fdx, _9l.fnm, _9l.nvd, _9l.nvm, _9l.si, _9l_Lucene50_0.doc, _9l_Lucene50_0.tim, _9l_Lucene50_0.tip, _9m.fdt, _9m.fdx, _9m.fnm, _9m.nvd, _9m.nvm, _9m.si, _9m_Lucene50_0.doc, _9m_Lucene50_0.tim, _9m_Lucene50_0.tip, _9p.fdt, _9p.fdx, _9p.fnm, _9p.nvd, _9p.nvm, _9p.si, _9p_Lucene50_0.doc, _9p_Lucene50_0.tim, _9p_Lucene50_0.tip, _9q.fdt, _9q.fdx, _9q.fnm, _9q.nvd, _9q.nvm, _9q.si, _9q_Lucene50_0.doc, _9q_Lucene50_0.tim, _9q_Lucene50_0.tip, _9r.fdt, _9r.fdx, _9r.fnm, _9r.nvd, _9r.nvm, _9r.si, _9r_Lucene50_0.doc, _9r_Lucene50_0.tim, _9r_Lucene50_0.tip, _9s.fdt, _9s.fdx, _9s.fnm, _9s.nvd, _9s.nvm, _9s.si, _9s_Lucene50_0.doc, _9s_Lucene50_0.tim, _9s_Lucene50_0.tip, _9u.fdt, _9u.fdx, _9u.fnm, _9u.nvd, _9u.nvm, _9u.si, _9u_Lucene50_0.doc, _9u_Lucene50_0.tim, _9u_Lucene50_0.tip, _9v.fdt, _9v.fdx, _9v.fnm, _9v.nvd, _9v.nvm, _9v.si, _9v_Lucene50_0.doc, _9v_Lucene50_0.tim, _9v_Lucene50_0.tip, _9w.fdt, _9w.fdx, _9w.fnm, _9w.nvd, _9w.nvm, _9w.si, _9w_Lucene50_0.doc, _9w_Lucene50_0.tim, _9w_Lucene50_0.tip, _9z.fdt, _9z.fdx, _9z.fnm, _9z.nvd, _9z.nvm, _9z.si, _9z_Lucene50_0.doc, _9z_Lucene50_0.tim, _9z_Lucene50_0.tip, _a0.cfe, _a0.cfs, _a0.si, segments_2]}]> but was:<[{indexVersion=1502535335192,generation=2,filelist=[_97.fdt, _97.fdx, _97.fnm, _97.nvd, _97.nvm, _97.si, _97_Lucene50_0.doc, _97_Lucene50_0.tim, _97_Lucene50_0.tip, _99.fdt, _99.fdx, _99.fnm, _99.nvd, _99.nvm, _99.si, _99_Lucene50_0.doc, _99_Lucene50_0.tim, _99_Lucene50_0.tip, _9c.fdt, _9c.fdx, _9c.fnm, _9c.nvd, _9c.nvm, _9c.si, _9c_Lucene50_0.doc, _9c_Lucene50_0.tim, _9c_Lucene50_0.tip, _9l.fdt, _9l.fdx, _9l.fnm, _9l.nvd, _9l.nvm, _9l.si, _9l_Lucene50_0.doc, _9l_Lucene50_0.tim, _9l_Lucene50_0.tip, _9m.fdt, _9m.fdx, _9m.fnm, _9m.nvd, _9m.nvm, _9m.si, _9m_Lucene50_0.doc, _9m_Lucene50_0.tim, _9m_Lucene50_0.tip, _9p.fdt, _9p.fdx, _9p.fnm, _9p.nvd, _9p.nvm, _9p.si, _9p_Lucene50_0.doc, _9p_Lucene50_0.tim, _9p_Lucene50_0.tip, _9q.fdt, _9q.fdx, _9q.fnm, _9q.nvd, _9q.nvm, _9q.si, _9q_Lucene50_0.doc, _9q_Lucene50_0.tim, _9q_Lucene50_0.tip, _9r.fdt, _9r.fdx, _9r.fnm, _9r.nvd, _9r.nvm, _9r.si, _9r_Lucene50_0.doc, _9r_Lucene50_0.tim, _9r_Lucene50_0.tip, _9s.fdt, _9s.fdx, _9s.fnm, _9s.nvd, _9s.nvm, _9s.si, _9s_Lucene50_0.doc, _9s_Lucene50_0.tim, _9s_Lucene50_0.tip, _9u.fdt, _9u.fdx, _9u.fnm, _9u.nvd, _9u.nvm, _9u.si, _9u_Lucene50_0.doc, _9u_Lucene50_0.tim, _9u_Lucene50_0.tip, _9v.fdt, _9v.fdx, _9v.fnm, _9v.nvd, _9v.nvm, _9v.si, _9v_Lucene50_0.doc, _9v_Lucene50_0.tim, _9v_Lucene50_0.tip, _9w.fdt, _9w.fdx, _9w.fnm, _9w.nvd, _9w.nvm, _9w.si, _9w_Lucene50_0.doc, _9w_Lucene50_0.tim, _9w_Lucene50_0.tip, _9z.fdt, _9z.fdx, _9z.fnm, _9z.nvd, _9z.nvm, _9z.si, _9z_Lucene50_0.doc, _9z_Lucene50_0.tim, _9z_Lucene50_0.tip, _a0.cfe, _a0.cfs, _a0.si, segments_2]}, {indexVersion=1502535335192,generation=3,filelist=[_9m.fdt, _9m.fdx, _9m.fnm, _9m.nvd, _9m.nvm, _9m.si, _9m_Lucene50_0.doc, _9m_Lucene50_0.tim, _9m_Lucene50_0.tip, _9u.fdt, _9u.fdx, _9u.fnm, _9u.nvd, _9u.nvm, _9u.si, _9u_Lucene50_0.doc, _9u_Lucene50_0.tim, _9u_Lucene50_0.tip, _9w.fdt, _9w.fdx, _9w.fnm, _9w.nvd, _9w.nvm, _9w.si, _9w_Lucene50_0.doc, _9w_Lucene50_0.tim, _9w_Lucene50_0.tip, _9x.cfe, _9x.cfs, _9x.si, _9z.fdt, _9z.fdx, _9z.fnm, _9z.nvd, _9z.nvm, _9z.si, _9z_Lucene50_0.doc, _9z_Lucene50_0.tim, _9z_Lucene50_0.tip, _a0.cfe, _a0.cfs, _a0.si, segments_3]}]>
Stack Trace:
java.lang.AssertionError: expected:<[{indexVersion=1502535335192,generation=2,filelist=[_97.fdt, _97.fdx, _97.fnm, _97.nvd, _97.nvm, _97.si, _97_Lucene50_0.doc, _97_Lucene50_0.tim, _97_Lucene50_0.tip, _99.fdt, _99.fdx, _99.fnm, _99.nvd, _99.nvm, _99.si, _99_Lucene50_0.doc, _99_Lucene50_0.tim, _99_Lucene50_0.tip, _9c.fdt, _9c.fdx, _9c.fnm, _9c.nvd, _9c.nvm, _9c.si, _9c_Lucene50_0.doc, _9c_Lucene50_0.tim, _9c_Lucene50_0.tip, _9l.fdt, _9l.fdx, _9l.fnm, _9l.nvd, _9l.nvm, _9l.si, _9l_Lucene50_0.doc, _9l_Lucene50_0.tim, _9l_Lucene50_0.tip, _9m.fdt, _9m.fdx, _9m.fnm, _9m.nvd, _9m.nvm, _9m.si, _9m_Lucene50_0.doc, _9m_Lucene50_0.tim, _9m_Lucene50_0.tip, _9p.fdt, _9p.fdx, _9p.fnm, _9p.nvd, _9p.nvm, _9p.si, _9p_Lucene50_0.doc, _9p_Lucene50_0.tim, _9p_Lucene50_0.tip, _9q.fdt, _9q.fdx, _9q.fnm, _9q.nvd, _9q.nvm, _9q.si, _9q_Lucene50_0.doc, _9q_Lucene50_0.tim, _9q_Lucene50_0.tip, _9r.fdt, _9r.fdx, _9r.fnm, _9r.nvd, _9r.nvm, _9r.si, _9r_Lucene50_0.doc, _9r_Lucene50_0.tim, _9r_Lucene50_0.tip, _9s.fdt, _9s.fdx, _9s.fnm, _9s.nvd, _9s.nvm, _9s.si, _9s_Lucene50_0.doc, _9s_Lucene50_0.tim, _9s_Lucene50_0.tip, _9u.fdt, _9u.fdx, _9u.fnm, _9u.nvd, _9u.nvm, _9u.si, _9u_Lucene50_0.doc, _9u_Lucene50_0.tim, _9u_Lucene50_0.tip, _9v.fdt, _9v.fdx, _9v.fnm, _9v.nvd, _9v.nvm, _9v.si, _9v_Lucene50_0.doc, _9v_Lucene50_0.tim, _9v_Lucene50_0.tip, _9w.fdt, _9w.fdx, _9w.fnm, _9w.nvd, _9w.nvm, _9w.si, _9w_Lucene50_0.doc, _9w_Lucene50_0.tim, _9w_Lucene50_0.tip, _9z.fdt, _9z.fdx, _9z.fnm, _9z.nvd, _9z.nvm, _9z.si, _9z_Lucene50_0.doc, _9z_Lucene50_0.tim, _9z_Lucene50_0.tip, _a0.cfe, _a0.cfs, _a0.si, segments_2]}]> but was:<[{indexVersion=1502535335192,generation=2,filelist=[_97.fdt, _97.fdx, _97.fnm, _97.nvd, _97.nvm, _97.si, _97_Lucene50_0.doc, _97_Lucene50_0.tim, _97_Lucene50_0.tip, _99.fdt, _99.fdx, _99.fnm, _99.nvd, _99.nvm, _99.si, _99_Lucene50_0.doc, _99_Lucene50_0.tim, _99_Lucene50_0.tip, _9c.fdt, _9c.fdx, _9c.fnm, _9c.nvd, _9c.nvm, _9c.si, _9c_Lucene50_0.doc, _9c_Lucene50_0.tim, _9c_Lucene50_0.tip, _9l.fdt, _9l.fdx, _9l.fnm, _9l.nvd, _9l.nvm, _9l.si, _9l_Lucene50_0.doc, _9l_Lucene50_0.tim, _9l_Lucene50_0.tip, _9m.fdt, _9m.fdx, _9m.fnm, _9m.nvd, _9m.nvm, _9m.si, _9m_Lucene50_0.doc, _9m_Lucene50_0.tim, _9m_Lucene50_0.tip, _9p.fdt, _9p.fdx, _9p.fnm, _9p.nvd, _9p.nvm, _9p.si, _9p_Lucene50_0.doc, _9p_Lucene50_0.tim, _9p_Lucene50_0.tip, _9q.fdt, _9q.fdx, _9q.fnm, _9q.nvd, _9q.nvm, _9q.si, _9q_Lucene50_0.doc, _9q_Lucene50_0.tim, _9q_Lucene50_0.tip, _9r.fdt, _9r.fdx, _9r.fnm, _9r.nvd, _9r.nvm, _9r.si, _9r_Lucene50_0.doc, _9r_Lucene50_0.tim, _9r_Lucene50_0.tip, _9s.fdt, _9s.fdx, _9s.fnm, _9s.nvd, _9s.nvm, _9s.si, _9s_Lucene50_0.doc, _9s_Lucene50_0.tim, _9s_Lucene50_0.tip, _9u.fdt, _9u.fdx, _9u.fnm, _9u.nvd, _9u.nvm, _9u.si, _9u_Lucene50_0.doc, _9u_Lucene50_0.tim, _9u_Lucene50_0.tip, _9v.fdt, _9v.fdx, _9v.fnm, _9v.nvd, _9v.nvm, _9v.si, _9v_Lucene50_0.doc, _9v_Lucene50_0.tim, _9v_Lucene50_0.tip, _9w.fdt, _9w.fdx, _9w.fnm, _9w.nvd, _9w.nvm, _9w.si, _9w_Lucene50_0.doc, _9w_Lucene50_0.tim, _9w_Lucene50_0.tip, _9z.fdt, _9z.fdx, _9z.fnm, _9z.nvd, _9z.nvm, _9z.si, _9z_Lucene50_0.doc, _9z_Lucene50_0.tim, _9z_Lucene50_0.tip, _a0.cfe, _a0.cfs, _a0.si, segments_2]}, {indexVersion=1502535335192,generation=3,filelist=[_9m.fdt, _9m.fdx, _9m.fnm, _9m.nvd, _9m.nvm, _9m.si, _9m_Lucene50_0.doc, _9m_Lucene50_0.tim, _9m_Lucene50_0.tip, _9u.fdt, _9u.fdx, _9u.fnm, _9u.nvd, _9u.nvm, _9u.si, _9u_Lucene50_0.doc, _9u_Lucene50_0.tim, _9u_Lucene50_0.tip, _9w.fdt, _9w.fdx, _9w.fnm, _9w.nvd, _9w.nvm, _9w.si, _9w_Lucene50_0.doc, _9w_Lucene50_0.tim, _9w_Lucene50_0.tip, _9x.cfe, _9x.cfs, _9x.si, _9z.fdt, _9z.fdx, _9z.fnm, _9z.nvd, _9z.nvm, _9z.si, _9z_Lucene50_0.doc, _9z_Lucene50_0.tim, _9z_Lucene50_0.tip, _a0.cfe, _a0.cfs, _a0.si, segments_3]}]>
at __randomizedtesting.SeedInfo.seed([8DBA7CAC21166B83:A86D679C515E6580]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:147)
at org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload(TestReplicationHandler.java:1277)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.update.AutoCommitTest.testMaxDocs
Error Message:
Exception during query
Stack Trace:
java.lang.RuntimeException: Exception during query
at __randomizedtesting.SeedInfo.seed([8DBA7CAC21166B83:343BAA730DFC6F09]:0)
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:886)
at org.apache.solr.update.AutoCommitTest.testMaxDocs(AutoCommitTest.java:225)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
</response>
request was:q=id:14&qt=&start=0&rows=20&version=2.2
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:879)
... 40 more
FAILED: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test
Error Message:
Timeout waiting for all live and active
Stack Trace:
java.lang.AssertionError: Timeout waiting for all live and active
at __randomizedtesting.SeedInfo.seed([8DBA7CAC21166B83:5EE43768FEA067B]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.testBasics(SharedFSAutoReplicaFailoverTest.java:246)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test(SharedFSAutoReplicaFailoverTest.java:136)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 8186 lines...]
[junit4] Suite: org.apache.lucene.search.TestInetAddressRangeQueries
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestInetAddressRangeQueries -Dtests.method=testRandomBig -Dtests.seed=4B58E77EC66368F0 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=ms-MY -Dtests.timezone=America/Grenada -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 1330s J1 | TestInetAddressRangeQueries.testRandomBig <<<
[junit4] > Throwable #1: java.lang.OutOfMemoryError: GC overhead limit exceeded
[junit4] > at __randomizedtesting.SeedInfo.seed([4B58E77EC66368F0:CC0F9AF1573A1470]:0)
[junit4] > at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectField$IntArrayWriter.get(DirectPostingsFormat.java:295)
[junit4] > at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectField.<init>(DirectPostingsFormat.java:413)
[junit4] > at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectFields.<init>(DirectPostingsFormat.java:129)
[junit4] > at org.apache.lucene.codecs.memory.DirectPostingsFormat.fieldsProducer(DirectPostingsFormat.java:113)
[junit4] > at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:292)
[junit4] > at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:372)
[junit4] > at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:112)
[junit4] > at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:78)
[junit4] > at org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:702)
[junit4] > at org.apache.lucene.util.TestUtil.checkIndex(TestUtil.java:300)
[junit4] > at org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:865)
[junit4] > at org.apache.lucene.util.IOUtils.close(IOUtils.java:89)
[junit4] > at org.apache.lucene.util.IOUtils.close(IOUtils.java:76)
[junit4] > at org.apache.lucene.search.BaseRangeFieldQueryTestCase.verify(BaseRangeFieldQueryTestCase.java:293)
[junit4] > at org.apache.lucene.search.BaseRangeFieldQueryTestCase.doTestRandom(BaseRangeFieldQueryTestCase.java:160)
[junit4] > at org.apache.lucene.search.BaseRangeFieldQueryTestCase.testRandomBig(BaseRangeFieldQueryTestCase.java:75)
[junit4] > at org.apache.lucene.search.TestInetAddressRangeQueries.testRandomBig(TestInetAddressRangeQueries.java:81)
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/build/misc/test/J1/temp/lucene.search.TestInetAddressRangeQueries_4B58E77EC66368F0-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {id=PostingsFormat(name=Direct)}, docValues:{ipRangeField=DocValuesFormat(name=Lucene70), id=DocValuesFormat(name=Lucene70)}, maxPointsInLeafNode=604, maxMBSortInHeap=7.616515375437137, sim=RandomSimilarity(queryNorm=true): {}, locale=ms-MY, timezone=America/Grenada
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=256025200,total=317194240
[junit4] 2> NOTE: All tests run in this JVM: [TestInetAddressRangeQueries]
[junit4] Completed [12/13 (1!)] on J1 in 1362.65s, 4 tests, 1 error <<< FAILURES!
[...truncated 4 lines...]
[junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/build/misc/test/temp/junit4-J1-20170812_090531_7387703719454961821416.sysout
[junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
[junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
[junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/heapdumps/java_pid14503.hprof ...
[junit4] Heap dump file created [591807176 bytes in 3.366 secs]
[junit4] <<< JVM J1: EOF ----
[...truncated 3487 lines...]
[junit4] Suite: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_8DBA7CAC21166B83-001/init-core-data-001
[junit4] 2> 78985 WARN (SUITE-FullSolrCloudDistribCmdsTest-seed#[8DBA7CAC21166B83]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=7 numCloses=7
[junit4] 2> 78985 INFO (SUITE-FullSolrCloudDistribCmdsTest-seed#[8DBA7CAC21166B83]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 78998 INFO (SUITE-FullSolrCloudDistribCmdsTest-seed#[8DBA7CAC21166B83]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
[junit4] 2> 78998 INFO (SUITE-FullSolrCloudDistribCmdsTest-seed#[8DBA7CAC21166B83]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 78998 INFO (SUITE-FullSolrCloudDistribCmdsTest-seed#[8DBA7CAC21166B83]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 2> 79088 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 79094 INFO (Thread-670) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 79094 INFO (Thread-670) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 79127 ERROR (Thread-670) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 79194 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.a.s.c.ZkTestServer start zk server on port:38477
[junit4] 2> 79277 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 79279 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 79280 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 79280 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 79281 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 79282 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 79283 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 79284 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 79285 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 79285 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 79286 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 79288 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
[junit4] 2> 79418 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 79419 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@286b5fec{/,null,AVAILABLE}
[junit4] 2> 79421 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@64a15a4e{HTTP/1.1,[http/1.1]}{127.0.0.1:44395}
[junit4] 2> 79421 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.e.j.s.Server Started @82396ms
[junit4] 2> 79421 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_8DBA7CAC21166B83-001/tempDir-001/control/data, replicaType=NRT, hostContext=/, hostPort=44395, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_8DBA7CAC21166B83-001/control-001/cores}
[junit4] 2> 79422 ERROR (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 79422 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.1.0
[junit4] 2> 79422 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 79422 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 79422 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-12T10:21:22.227Z
[junit4] 2> 79437 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 79437 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_8DBA7CAC21166B83-001/control-001/solr.xml
[junit4] 2> 79515 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@1862939e, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 79518 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38477/solr
[junit4] 2> 79639 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [n:127.0.0.1:44395_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 79639 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [n:127.0.0.1:44395_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:44395_
[junit4] 2> 79640 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [n:127.0.0.1:44395_ ] o.a.s.c.Overseer Overseer (id=98470021162926084-127.0.0.1:44395_-n_0000000000) starting
[junit4] 2> 79701 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [n:127.0.0.1:44395_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44395_
[junit4] 2> 79703 INFO (zkCallback-106-thread-1-processing-n:127.0.0.1:44395_) [n:127.0.0.1:44395_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 79786 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [n:127.0.0.1:44395_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1862939e
[junit4] 2> 79789 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [n:127.0.0.1:44395_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1862939e
[junit4] 2> 79789 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [n:127.0.0.1:44395_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1862939e
[junit4] 2> 79791 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [n:127.0.0.1:44395_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_8DBA7CAC21166B83-001/control-001/cores
[junit4] 2> 79809 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 79809 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:38477/solr ready
[junit4] 2> 79822 INFO (qtp1916269609-1194) [n:127.0.0.1:44395_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:44395_&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 79831 INFO (OverseerThreadFactory-414-thread-1-processing-n:127.0.0.1:44395_) [n:127.0.0.1:44395_ ] o.a.s.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 79942 INFO (qtp1916269609-1196) [n:127.0.0.1:44395_ ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 79943 INFO (qtp1916269609-1196) [n:127.0.0.1:44395_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 80052 INFO (zkCallback-106-thread-1-processing-n:127.0.0.1:44395_) [n:127.0.0.1:44395_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 80974 INFO (qtp1916269609-1196) [n:127.0.0.1:44395_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 81002 INFO (qtp1916269609-1196) [n:127.0.0.1:44395_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
[junit4] 2> 81316 INFO (qtp1916269609-1196) [n:127.0.0.1:44395_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 81392 INFO (qtp1916269609-1196) [n:127.0.0.1:44395_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
[junit4] 2> 81393 INFO (qtp1916269609-1196) [n:127.0.0.1:44395_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1862939e
[junit4] 2> 81393 INFO (qtp1916269609-1196) [n:127.0.0.1:44395_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 81393 INFO (qtp1916269609-1196) [n:127.0.0.1:44395_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_8DBA7CAC21166B83-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/../../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_8DBA7CAC21166B83-001/control-001/cores/control_collection_shard1_replica_n1/data/]
[junit4] 2> 81401 INFO (qtp1916269609-1196) [n:127.0.0.1:44395_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=36, maxMergeAtOnceExplicit=41, maxMergedSegmentMB=1.96484375, floorSegmentMB=1.1796875, forceMergeDeletesPctAllowed=13.045785781309283, segmentsPerTier=12.0, maxCFSSegmentSizeMB=1.12890625, noCFSRatio=0.5459606932158686
[junit4] 2> 81420 WARN (qtp1916269609-1196) [n:127.0.0.1:44395_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 81594 INFO (qtp1916269609-1196) [n:127.0.0.1:44395_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 81594 INFO (qtp1916269609-1196) [n:127.0.0.1:44395_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 81595 INFO (qtp1916269609-1196) [n:127.0.0.1:44395_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 81595 INFO (qtp1916269609-1196) [n:127.0.0.1:44395_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 81596 INFO (qtp1916269609-1196) [n:127.0.0.1:44395_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=11, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7432232716553033]
[junit4] 2> 81597 INFO (qtp1916269609-1196) [n:127.0.0.1:44395_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@ad0d7bf[control_collection_shard1_replica_n1] main]
[junit4] 2> 81598 INFO (qtp1916269609-1196) [n:127.0.0.1:44395_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 81598 INFO (qtp1916269609-1196) [n:127.0.0.1:44395_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 81601 INFO (qtp1916269609-1196) [n:127.0.0.1:44395_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 81601 INFO (qtp1916269609-1196) [n:127.0.0.1:44395_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1575520341229305856
[junit4] 2> 81615 INFO (searcherExecutor-417-thread-1-processing-n:127.0.0.1:44395_ x:control_collection_shard1_replica_n1 s:shard1 c:control_collection) [n:127.0.0.1:44395_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@ad0d7bf[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 82038 INFO (qtp1916269609-1196) [n:127.0.0.1:44395_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 82038 INFO (qtp1916269609-1196) [n:127.0.0.1:44395_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 82038 INFO (qtp1916269609-1196) [n:127.0.0.1:44395_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:44395/control_collection_shard1_replica_n1/
[junit4] 2> 82038 INFO (qtp1916269609-1196) [n:127.0.0.1:44395_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 82038 INFO (qtp1916269609-1196) [n:127.0.0.1:44395_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:44395/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 82038 INFO (qtp1916269609-1196) [n:127.0.0.1:44395_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 82041 INFO (qtp1916269609-1196) [n:127.0.0.1:44395_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:44395/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 82143 INFO (zkCallback-106-thread-1-processing-n:127.0.0.1:44395_) [n:127.0.0.1:44395_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 82254 INFO (qtp1916269609-1196) [n:127.0.0.1:44395_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 82263 INFO (qtp1916269609-1196) [n:127.0.0.1:44395_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2320
[junit4] 2> 82270 INFO (qtp1916269609-1194) [n:127.0.0.1:44395_ ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 82366 INFO (zkCallback-106-thread-1-processing-n:127.0.0.1:44395_) [n:127.0.0.1:44395_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 83270 INFO (qtp1916269609-1194) [n:127.0.0.1:44395_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:44395_&wt=javabin&version=2} status=0 QTime=3448
[junit4] 2> 83372 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 83373 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:38477/solr ready
[junit4] 2> 83374 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 83376 INFO (qtp1916269609-1195) [n:127.0.0.1:44395_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=3&createNodeSet=&stateFormat=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 83378 INFO (OverseerThreadFactory-414-thread-2-processing-n:127.0.0.1:44395_) [n:127.0.0.1:44395_ ] o.a.s.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 83378 INFO (OverseerCollectionConfigSetProcessor-98470021162926084-127.0.0.1:44395_-n_0000000000) [n:127.0.0.1:44395_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 83379 WARN (OverseerThreadFactory-414-thread-2-processing-n:127.0.0.1:44395_) [n:127.0.0.1:44395_ ] o.a.s.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
[junit4] 2> 83587 INFO (qtp1916269609-1195) [n:127.0.0.1:44395_ ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 83587 INFO (qtp1916269609-1195) [n:127.0.0.1:44395_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=3&createNodeSet=&stateFormat=1&wt=javabin&version=2} status=0 QTime=211
[junit4] 2> 83756 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_8DBA7CAC21166B83-001/shard-1-001 of type NRT
[junit4] 2> 83757 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 83862 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@64777c80{/,null,AVAILABLE}
[junit4] 2> 83863 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@5b606a7e{HTTP/1.1,[http/1.1]}{127.0.0.1:59892}
[junit4] 2> 83863 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.e.j.s.Server Started @86838ms
[junit4] 2> 83863 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_8DBA7CAC21166B83-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=59892, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_8DBA7CAC21166B83-001/shard-1-001/cores}
[junit4] 2> 83863 ERROR (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 83863 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.1.0
[junit4] 2> 83863 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 83863 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 83863 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-12T10:21:26.668Z
[junit4] 2> 83919 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 83920 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_8DBA7CAC21166B83-001/shard-1-001/solr.xml
[junit4] 2> 83924 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@1862939e, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 83942 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38477/solr
[junit4] 2> 83993 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [n:127.0.0.1:59892_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 83994 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [n:127.0.0.1:59892_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 83999 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [n:127.0.0.1:59892_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59892_
[junit4] 2> 84000 INFO (zkCallback-113-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 84000 INFO (zkCallback-106-thread-1-processing-n:127.0.0.1:44395_) [n:127.0.0.1:44395_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 84009 INFO (zkCallback-118-thread-1-processing-n:127.0.0.1:59892_) [n:127.0.0.1:59892_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 84115 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [n:127.0.0.1:59892_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1862939e
[junit4] 2> 84118 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [n:127.0.0.1:59892_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1862939e
[junit4] 2> 84119 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [n:127.0.0.1:59892_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1862939e
[junit4] 2> 84120 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[8DBA7CAC21166B83]) [n:127.0.0.1:59892_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_8DBA7CAC21166B83-001/shard-1-001/cores
[junit4] 2> 84141 INFO (qtp1916269609-1191) [n:127.0.0.1:44395_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:59892_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 84154 INFO (OverseerCollectionConfigSetProcessor-98470021162926084-127.0.0.1:44395_-n_0000000000) [n:127.0
[...truncated too long message...]
edMLTComponentTest, CursorPagingTest, TestCrossCoreJoin, TestCursorMarkWithoutUniqueKey, TestHighlightDedupGrouping, TestSimpleTrackingShardHandler, ConnectionReuseTest, ActionThrottleTest, AliasIntegrationTest, CdcrBootstrapTest, CdcrReplicationDistributedZkTest, CdcrReplicationHandlerTest, CdcrVersionReplicationTest, ChaosMonkeyNothingIsSafeWithPullReplicasTest, CleanupOldIndexTest, CollectionStateFormat2Test, CollectionTooManyReplicasTest, CollectionsAPISolrJTest, ConcurrentDeleteAndCreateCollectionTest, ConnectionManagerTest, CreateCollectionCleanupTest, DeleteLastCustomShardedReplicaTest, DeleteNodeTest, DeleteShardTest, DeleteStatusTest, DistribJoinFromCollectionTest, DocValuesNotIndexedTest, HttpPartitionTest, OverseerModifyCollectionTest, OverseerRolesTest, OverseerStatusTest, RemoteQueryErrorTest, RestartWhileUpdatingTest, SharedFSAutoReplicaFailoverTest, SharedFSAutoReplicaFailoverUtilsTest, TestCloudDeleteByQuery, TestCloudJSONFacetJoinDomain, TestCloudSearcherWarming, TestCollectionsAPIViaSolrCloudCluster, TestConfigSetsAPIZkFailure, TestDeleteCollectionOnDownNodes, TestDistribDocBasedVersion, TestDownShardTolerantSearch, TestHdfsCloudBackupRestore, TestMiniSolrCloudClusterSSL, TestRandomFlRTGCloud, TestSSLRandomization, TestSolrCloudWithDelegationTokens, TestTlogReplica, HdfsChaosMonkeySafeLeaderTest]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=HdfsChaosMonkeySafeLeaderTest -Dtests.seed=8DBA7CAC21166B83 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=th-TH -Dtests.timezone=EST -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J2 | HdfsChaosMonkeySafeLeaderTest (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest:
[junit4] > 1) Thread[id=36221, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-HdfsChaosMonkeySafeLeaderTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135)
[junit4] > 2) Thread[id=36222, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-HdfsChaosMonkeySafeLeaderTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135)
[junit4] > at __randomizedtesting.SeedInfo.seed([8DBA7CAC21166B83]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=36221, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-HdfsChaosMonkeySafeLeaderTest]
[junit4] > at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:328)
[junit4] > at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
[junit4] > at org.apache.solr.common.util.JavaBinCodec.writeSolrInputDocument(JavaBinCodec.java:616)
[junit4] > at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:355)
[junit4] > at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
[junit4] > at org.apache.solr.common.util.JavaBinCodec.writeMapEntry(JavaBinCodec.java:764)
[junit4] > at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:383)
[junit4] > at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
[junit4] > at org.apache.solr.common.util.JavaBinCodec.writeIterator(JavaBinCodec.java:705)
[junit4] > at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:367)
[junit4] > at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
[junit4] > at org.apache.solr.common.util.JavaBinCodec.writeNamedList(JavaBinCodec.java:223)
[junit4] > at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:330)
[junit4] > at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
[junit4] > at org.apache.solr.common.util.JavaBinCodec.marshal(JavaBinCodec.java:155)
[junit4] > at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.marshal(JavaBinUpdateRequestCodec.java:91)
[junit4] > at org.apache.solr.client.solrj.impl.BinaryRequestWriter.getContentStream(BinaryRequestWriter.java:67)
[junit4] > at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:94)
[junit4] > at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:104)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:382)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
[junit4] > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
[junit4] > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195)
[junit4] > at org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:176)
[junit4] > at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:115)
[junit4] > 2) Thread[id=36222, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-HdfsChaosMonkeySafeLeaderTest]
[junit4] > at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:328)
[junit4] > at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
[junit4] > at org.apache.solr.common.util.JavaBinCodec.writeMapEntry(JavaBinCodec.java:764)
[junit4] > at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:383)
[junit4] > at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
[junit4] > at org.apache.solr.common.util.JavaBinCodec.writeIterator(JavaBinCodec.java:705)
[junit4] > at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:367)
[junit4] > at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
[junit4] > at org.apache.solr.common.util.JavaBinCodec.writeNamedList(JavaBinCodec.java:223)
[junit4] > at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:330)
[junit4] > at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:228)
[junit4] > at org.apache.solr.common.util.JavaBinCodec.marshal(JavaBinCodec.java:155)
[junit4] > at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.marshal(JavaBinUpdateRequestCodec.java:91)
[junit4] > at org.apache.solr.client.solrj.impl.BinaryRequestWriter.getContentStream(BinaryRequestWriter.java:67)
[junit4] > at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getDelegate(RequestWriter.java:94)
[junit4] > at org.apache.solr.client.solrj.request.RequestWriter$LazyContentStream.getName(RequestWriter.java:104)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:382)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
[junit4] > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
[junit4] > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195)
[junit4] > at org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:176)
[junit4] > at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:115)
[junit4] > at __randomizedtesting.SeedInfo.seed([8DBA7CAC21166B83]:0)
[junit4] Completed [556/728 (8!)] on J2 in 101.60s, 1 test, 1 failure, 2 errors <<< FAILURES!
[...truncated 508 lines...]
[junit4] JVM J2: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/temp/junit4-J2-20170812_101959_7844560544734125426757.sysout
[junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
[junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
[junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/heapdumps/java_pid5908.hprof ...
[junit4] Heap dump file created [642478382 bytes in 2.821 secs]
[junit4] <<< JVM J2: EOF ----
[...truncated 7520 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/build.xml:818: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/build.xml:762: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid14503.hprof
* java_pid5908.hprof
Total time: 289 minutes 8 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any