You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2017/11/05 21:43:06 UTC
[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1414 - Still
unstable
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1414/
3 tests failed.
FAILED: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.test
Error Message:
The Monkey ran for over 45 seconds and no jetties were stopped - this is worth investigating!
Stack Trace:
java.lang.AssertionError: The Monkey ran for over 45 seconds and no jetties were stopped - this is worth investigating!
at __randomizedtesting.SeedInfo.seed([F95A06AB89972DF5:710E3971276B400D]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.ChaosMonkey.stopTheMonkey(ChaosMonkey.java:587)
at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.test(ChaosMonkeySafeLeaderTest.java:133)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
Error Message:
2 threads leaked from SUITE scope at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest: 1) Thread[id=4556, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderTest] at java.lang.Thread.sleep(Native Method) at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135) 2) Thread[id=4557, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderTest] at java.lang.Thread.sleep(Native Method) at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest:
1) Thread[id=4556, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135)
2) Thread[id=4557, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135)
at __randomizedtesting.SeedInfo.seed([F95A06AB89972DF5]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=4556, name=StoppableIndexingThread, state=BLOCKED, group=TGRP-ChaosMonkeySafeLeaderTest] at java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006) at org.apache.http.protocol.BasicHttpContext.setAttribute(BasicHttpContext.java:75) at org.apache.http.protocol.HttpCoreContext.setAttribute(HttpCoreContext.java:107) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:180) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:542) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255) at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:183) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:200) at org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:176) at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:115) 2) Thread[id=4557, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderTest] at java.io.PrintStream.println(PrintStream.java:805) at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:120)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=4556, name=StoppableIndexingThread, state=BLOCKED, group=TGRP-ChaosMonkeySafeLeaderTest]
at java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006)
at org.apache.http.protocol.BasicHttpContext.setAttribute(BasicHttpContext.java:75)
at org.apache.http.protocol.HttpCoreContext.setAttribute(HttpCoreContext.java:107)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:180)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:542)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:183)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:200)
at org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:176)
at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:115)
2) Thread[id=4557, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderTest]
at java.io.PrintStream.println(PrintStream.java:805)
at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:120)
at __randomizedtesting.SeedInfo.seed([F95A06AB89972DF5]:0)
Build Log:
[...truncated 12906 lines...]
[junit4] Suite: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_F95A06AB89972DF5-001/init-core-data-001
[junit4] 2> 858779 WARN (SUITE-ChaosMonkeySafeLeaderTest-seed#[F95A06AB89972DF5]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=21 numCloses=21
[junit4] 2> 858779 INFO (SUITE-ChaosMonkeySafeLeaderTest-seed#[F95A06AB89972DF5]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 858781 INFO (SUITE-ChaosMonkeySafeLeaderTest-seed#[F95A06AB89972DF5]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 858781 INFO (SUITE-ChaosMonkeySafeLeaderTest-seed#[F95A06AB89972DF5]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 858781 INFO (SUITE-ChaosMonkeySafeLeaderTest-seed#[F95A06AB89972DF5]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 2> 858800 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 858800 INFO (Thread-500) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 858800 INFO (Thread-500) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 858801 ERROR (Thread-500) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 858904 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.ZkTestServer start zk server on port:35937
[junit4] 2> 859132 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 859134 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 859135 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 859160 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 859161 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 859162 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 859162 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 859163 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 859164 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 859164 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 859177 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 859178 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
[junit4] 2> 860593 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 860597 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2b0383f1{/,null,AVAILABLE}
[junit4] 2> 860598 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@11becf32{HTTP/1.1,[http/1.1]}{127.0.0.1:34207}
[junit4] 2> 860598 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.e.j.s.Server Started @873445ms
[junit4] 2> 860598 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_F95A06AB89972DF5-001/tempDir-001/control/data, replicaType=NRT, hostContext=/, hostPort=34207, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_F95A06AB89972DF5-001/control-001/cores}
[junit4] 2> 860598 ERROR (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 860598 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 860598 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 860598 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 860598 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-11-05T19:05:54.391Z
[junit4] 2> 860623 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 860623 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_F95A06AB89972DF5-001/control-001/solr.xml
[junit4] 2> 860627 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 860627 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 860670 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@1adc8034, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 860685 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35937/solr
[junit4] 2> 860998 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [n:127.0.0.1:34207_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 860999 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [n:127.0.0.1:34207_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:34207_
[junit4] 2> 860999 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [n:127.0.0.1:34207_ ] o.a.s.c.Overseer Overseer (id=98953380010000388-127.0.0.1:34207_-n_0000000000) starting
[junit4] 2> 861163 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [n:127.0.0.1:34207_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34207_
[junit4] 2> 861198 INFO (zkCallback-1743-thread-1-processing-n:127.0.0.1:34207_) [n:127.0.0.1:34207_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 861601 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [n:127.0.0.1:34207_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1adc8034
[junit4] 2> 861769 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [n:127.0.0.1:34207_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1adc8034
[junit4] 2> 861769 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [n:127.0.0.1:34207_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1adc8034
[junit4] 2> 861821 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [n:127.0.0.1:34207_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_F95A06AB89972DF5-001/control-001/cores
[junit4] 2> 861949 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 861949 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:35937/solr ready
[junit4] 2> 861951 INFO (qtp74169703-4121) [n:127.0.0.1:34207_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:34207_&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 861953 INFO (OverseerThreadFactory-584-thread-1-processing-n:127.0.0.1:34207_) [n:127.0.0.1:34207_ ] o.a.s.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 862088 INFO (qtp74169703-4119) [n:127.0.0.1:34207_ ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 862088 INFO (qtp74169703-4119) [n:127.0.0.1:34207_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 862210 INFO (zkCallback-1743-thread-1-processing-n:127.0.0.1:34207_) [n:127.0.0.1:34207_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 863262 INFO (qtp74169703-4119) [n:127.0.0.1:34207_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 863297 INFO (qtp74169703-4119) [n:127.0.0.1:34207_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
[junit4] 2> 863703 INFO (qtp74169703-4119) [n:127.0.0.1:34207_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 863786 INFO (qtp74169703-4119) [n:127.0.0.1:34207_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
[junit4] 2> 863786 INFO (qtp74169703-4119) [n:127.0.0.1:34207_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1adc8034
[junit4] 2> 863857 INFO (qtp74169703-4119) [n:127.0.0.1:34207_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 863857 INFO (qtp74169703-4119) [n:127.0.0.1:34207_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_F95A06AB89972DF5-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_F95A06AB89972DF5-001/control-001/cores/control_collection_shard1_replica_n1/data/]
[junit4] 2> 863874 INFO (qtp74169703-4119) [n:127.0.0.1:34207_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=46, maxMergeAtOnceExplicit=10, maxMergedSegmentMB=82.958984375, floorSegmentMB=0.96875, forceMergeDeletesPctAllowed=7.808536689504667, segmentsPerTier=16.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7222890085757693
[junit4] 2> 863931 WARN (qtp74169703-4119) [n:127.0.0.1:34207_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 864133 INFO (qtp74169703-4119) [n:127.0.0.1:34207_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 864133 INFO (qtp74169703-4119) [n:127.0.0.1:34207_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 864134 INFO (qtp74169703-4119) [n:127.0.0.1:34207_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 15000ms;
[junit4] 2> 864134 INFO (qtp74169703-4119) [n:127.0.0.1:34207_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 864136 INFO (qtp74169703-4119) [n:127.0.0.1:34207_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=26, maxMergeAtOnceExplicit=50, maxMergedSegmentMB=32.5908203125, floorSegmentMB=0.951171875, forceMergeDeletesPctAllowed=6.617594327471268, segmentsPerTier=7.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 864214 INFO (qtp74169703-4119) [n:127.0.0.1:34207_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@c874d62[control_collection_shard1_replica_n1] main]
[junit4] 2> 864215 INFO (qtp74169703-4119) [n:127.0.0.1:34207_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 864215 INFO (qtp74169703-4119) [n:127.0.0.1:34207_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 864216 INFO (qtp74169703-4119) [n:127.0.0.1:34207_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 864216 INFO (qtp74169703-4119) [n:127.0.0.1:34207_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1583254085838045184
[junit4] 2> 864288 INFO (searcherExecutor-587-thread-1-processing-n:127.0.0.1:34207_ x:control_collection_shard1_replica_n1 s:shard1 c:control_collection) [n:127.0.0.1:34207_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@c874d62[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 864302 INFO (qtp74169703-4119) [n:127.0.0.1:34207_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 864302 INFO (qtp74169703-4119) [n:127.0.0.1:34207_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 864302 INFO (qtp74169703-4119) [n:127.0.0.1:34207_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:34207/control_collection_shard1_replica_n1/
[junit4] 2> 864302 INFO (qtp74169703-4119) [n:127.0.0.1:34207_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 864302 INFO (qtp74169703-4119) [n:127.0.0.1:34207_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:34207/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 864302 INFO (qtp74169703-4119) [n:127.0.0.1:34207_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 864333 INFO (qtp74169703-4119) [n:127.0.0.1:34207_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:34207/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 864340 INFO (zkCallback-1743-thread-2-processing-n:127.0.0.1:34207_) [n:127.0.0.1:34207_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 864383 INFO (qtp74169703-4119) [n:127.0.0.1:34207_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 864385 INFO (qtp74169703-4119) [n:127.0.0.1:34207_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2297
[junit4] 2> 864398 INFO (qtp74169703-4121) [n:127.0.0.1:34207_ ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 864498 INFO (zkCallback-1743-thread-3-processing-n:127.0.0.1:34207_) [n:127.0.0.1:34207_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 865401 INFO (qtp74169703-4121) [n:127.0.0.1:34207_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:34207_&wt=javabin&version=2} status=0 QTime=3450
[junit4] 2> 865403 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15f8d9328d20005, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 865435 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 865436 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:35937/solr ready
[junit4] 2> 865436 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 865437 INFO (qtp74169703-4125) [n:127.0.0.1:34207_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&stateFormat=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 865473 INFO (OverseerCollectionConfigSetProcessor-98953380010000388-127.0.0.1:34207_-n_0000000000) [n:127.0.0.1:34207_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 865473 INFO (OverseerThreadFactory-584-thread-2-processing-n:127.0.0.1:34207_) [n:127.0.0.1:34207_ ] o.a.s.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 865496 WARN (OverseerThreadFactory-584-thread-2-processing-n:127.0.0.1:34207_) [n:127.0.0.1:34207_ ] o.a.s.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
[junit4] 2> 865712 INFO (qtp74169703-4125) [n:127.0.0.1:34207_ ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 865713 INFO (qtp74169703-4125) [n:127.0.0.1:34207_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&stateFormat=1&wt=javabin&version=2} status=0 QTime=275
[junit4] 2> 867122 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_F95A06AB89972DF5-001/shard-1-001 of type NRT
[junit4] 2> 867122 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 867146 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4c4e508e{/,null,AVAILABLE}
[junit4] 2> 867146 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7cb6fa6f{HTTP/1.1,[http/1.1]}{127.0.0.1:37078}
[junit4] 2> 867146 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.e.j.s.Server Started @879993ms
[junit4] 2> 867146 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_F95A06AB89972DF5-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=37078, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_F95A06AB89972DF5-001/shard-1-001/cores}
[junit4] 2> 867147 ERROR (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 867147 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 867147 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 867147 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 867147 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-11-05T19:06:00.940Z
[junit4] 2> 867340 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 867340 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_F95A06AB89972DF5-001/shard-1-001/solr.xml
[junit4] 2> 867343 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 867343 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 867357 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@1adc8034, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 867385 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35937/solr
[junit4] 2> 867438 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [n:127.0.0.1:37078_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 867440 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [n:127.0.0.1:37078_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 867454 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [n:127.0.0.1:37078_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37078_
[junit4] 2> 867492 INFO (zkCallback-1756-thread-1-processing-n:127.0.0.1:37078_) [n:127.0.0.1:37078_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 867494 INFO (zkCallback-1743-thread-3-processing-n:127.0.0.1:34207_) [n:127.0.0.1:34207_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 867495 INFO (zkCallback-1751-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 867530 INFO (OverseerCollectionConfigSetProcessor-98953380010000388-127.0.0.1:34207_-n_0000000000) [n:127.0.0.1:34207_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 867838 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [n:127.0.0.1:37078_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1adc8034
[junit4] 2> 867908 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [n:127.0.0.1:37078_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1adc8034
[junit4] 2> 867908 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [n:127.0.0.1:37078_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1adc8034
[junit4] 2> 867909 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [n:127.0.0.1:37078_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_F95A06AB89972DF5-001/shard-1-001/cores
[junit4] 2> 868175 INFO (qtp74169703-4118) [n:127.0.0.1:34207_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:37078_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 868224 INFO (OverseerThreadFactory-584-thread-3-processing-n:127.0.0.1:34207_) [n:127.0.0.1:34207_ ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:37078_ for creating new replica
[junit4] 2> 868261 INFO (qtp506102776-4171) [n:127.0.0.1:37078_ ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n21&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 868261 INFO (qtp506102776-4171) [n:127.0.0.1:37078_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 869307 INFO (qtp506102776-4171) [n:127.0.0.1:37078_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 869379 INFO (qtp506102776-4171) [n:127.0.0.1:37078_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.s.IndexSchema [collection1_shard1_replica_n21] Schema name=test
[junit4] 2> 869994 INFO (qtp506102776-4171) [n:127.0.0.1:37078_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 870085 INFO (qtp506102776-4171) [n:127.0.0.1:37078_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n21' using configuration from collection collection1, trusted=true
[junit4] 2> 870086 INFO (qtp506102776-4171) [n:127.0.0.1:37078_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n21' (registry 'solr.core.collection1.shard1.replica_n21') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1adc8034
[junit4] 2> 870086 INFO (qtp506102776-4171) [n:127.0.0.1:37078_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 870086 INFO (qtp506102776-4171) [n:127.0.0.1:37078_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SolrCore [[collection1_shard1_replica_n21] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_F95A06AB89972DF5-001/shard-1-001/cores/collection1_shard1_replica_n21], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_F95A06AB89972DF5-001/shard-1-001/cores/collection1_shard1_replica_n21/data/]
[junit4] 2> 870102 INFO (qtp506102776-4171) [n:127.0.0.1:37078_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=46, maxMergeAtOnceExplicit=10, maxMergedSegmentMB=82.958984375, floorSegmentMB=0.96875, forceMergeDeletesPctAllowed=7.808536689504667, segmentsPerTier=16.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7222890085757693
[junit4] 2> 870184 WARN (qtp506102776-4171) [n:127.0.0.1:37078_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 870470 INFO (qtp506102776-4171) [n:127.0.0.1:37078_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 870470 INFO (qtp506102776-4171) [n:127.0.0.1:37078_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 870471 INFO (qtp506102776-4171) [n:127.0.0.1:37078_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 15000ms;
[junit4] 2> 870471 INFO (qtp506102776-4171) [n:127.0.0.1:37078_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 870473 INFO (qtp506102776-4171) [n:127.0.0.1:37078_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=26, maxMergeAtOnceExplicit=50, maxMergedSegmentMB=32.5908203125, floorSegmentMB=0.951171875, forceMergeDeletesPctAllowed=6.617594327471268, segmentsPerTier=7.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 870473 INFO (qtp506102776-4171) [n:127.0.0.1:37078_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.s.SolrIndexSearcher Opening [Searcher@34ef143c[collection1_shard1_replica_n21] main]
[junit4] 2> 870474 INFO (qtp506102776-4171) [n:127.0.0.1:37078_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 870507 INFO (qtp506102776-4171) [n:127.0.0.1:37078_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 870508 INFO (qtp506102776-4171) [n:127.0.0.1:37078_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 870509 INFO (searcherExecutor-598-thread-1-processing-n:127.0.0.1:37078_ x:collection1_shard1_replica_n21 s:shard1 c:collection1) [n:127.0.0.1:37078_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SolrCore [collection1_shard1_replica_n21] Registered new searcher Searcher@34ef143c[collection1_shard1_replica_n21] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 870509 INFO (qtp506102776-4171) [n:127.0.0.1:37078_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1583254092436733952
[junit4] 2> 870566 INFO (qtp506102776-4171) [n:127.0.0.1:37078_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 870566 INFO (qtp506102776-4171) [n:127.0.0.1:37078_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 870566 INFO (qtp506102776-4171) [n:127.0.0.1:37078_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:37078/collection1_shard1_replica_n21/
[junit4] 2> 870566 INFO (qtp506102776-4171) [n:127.0.0.1:37078_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 870566 INFO (qtp506102776-4171) [n:127.0.0.1:37078_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SyncStrategy http://127.0.0.1:37078/collection1_shard1_replica_n21/ has no replicas
[junit4] 2> 870566 INFO (qtp506102776-4171) [n:127.0.0.1:37078_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 870580 INFO (qtp506102776-4171) [n:127.0.0.1:37078_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:37078/collection1_shard1_replica_n21/ shard1
[junit4] 2> 870637 INFO (qtp506102776-4171) [n:127.0.0.1:37078_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 870638 INFO (qtp506102776-4171) [n:127.0.0.1:37078_ c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n21&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2377
[junit4] 2> 870655 INFO (qtp74169703-4118) [n:127.0.0.1:34207_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:37078_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=2480
[junit4] 2> 871256 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_F95A06AB89972DF5-001/shard-2-001 of type NRT
[junit4] 2> 871257 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 871303 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@53668c12{/,null,AVAILABLE}
[junit4] 2> 871303 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@a2365ce{HTTP/1.1,[http/1.1]}{127.0.0.1:46148}
[junit4] 2> 871303 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.e.j.s.Server Started @884150ms
[junit4] 2> 871303 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_F95A06AB89972DF5-001/tempDir-001/jetty2, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/, hostPort=46148, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_F95A06AB89972DF5-001/shard-2-001/cores}
[junit4] 2> 871303 ERROR (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 871316 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 871316 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 871317 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 871317 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-11-05T19:06:05.110Z
[junit4] 2> 871348 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 871348 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_F95A06AB89972DF5-001/shard-2-001/solr.xml
[junit4] 2> 871351 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 871351 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 871352 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@1adc8034, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 871355 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35937/solr
[junit4] 2> 871437 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [n:127.0.0.1:46148_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 871438 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [n:127.0.0.1:46148_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 871440 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [n:127.0.0.1:46148_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46148_
[junit4] 2> 871453 INFO (zkCallback-1756-thread-1-processing-n:127.0.0.1:37078_) [n:127.0.0.1:37078_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 871490 INFO (zkCallback-1762-thread-1-processing-n:127.0.0.1:46148_) [n:127.0.0.1:46148_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 871492 INFO (zkCallback-1743-thread-3-processing-n:127.0.0.1:34207_) [n:127.0.0.1:34207_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 871494 INFO (zkCallback-1751-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 871820 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [n:127.0.0.1:46148_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1adc8034
[junit4] 2> 871875 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [n:127.0.0.1:46148_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1adc8034
[junit4] 2> 871875 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [n:127.0.0.1:46148_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1adc8034
[junit4] 2> 871893 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [n:127.0.0.1:46148_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_F95A06AB89972DF5-001/shard-2-001/cores
[junit4] 2> 872263 INFO (OverseerCollectionConfigSetProcessor-98953380010000388-127.0.0.1:34207_-n_0000000000) [n:127.0.0.1:34207_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 872277 INFO (qtp74169703-4119) [n:127.0.0.1:34207_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:46148_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 872321 INFO (OverseerThreadFactory-584-thread-4-processing-n:127.0.0.1:34207_) [n:127.0.0.1:34207_ ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:46148_ for creating new replica
[junit4] 2> 872410 INFO (qtp1449866319-4198) [n:127.0.0.1:46148_ ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n23&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 872410 INFO (qtp1449866319-4198) [n:127.0.0.1:46148_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 873559 INFO (qtp1449866319-4198) [n:127.0.0.1:46148_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 873598 INFO (qtp1449866319-4198) [n:127.0.0.1:46148_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.s.IndexSchema [collection1_shard1_replica_n23] Schema name=test
[junit4] 2> 874081 INFO (qtp1449866319-4198) [n:127.0.0.1:46148_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 874151 INFO (qtp1449866319-4198) [n:127.0.0.1:46148_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n23' using configuration from collection collection1, trusted=true
[junit4] 2> 874184 INFO (qtp1449866319-4198) [n:127.0.0.1:46148_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n23' (registry 'solr.core.collection1.shard1.replica_n23') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1adc8034
[junit4] 2> 874184 INFO (qtp1449866319-4198) [n:127.0.0.1:46148_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 874185 INFO (qtp1449866319-4198) [n:127.0.0.1:46148_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.c.SolrCore [[collection1_shard1_replica_n23] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_F95A06AB89972DF5-001/shard-2-001/cores/collection1_shard1_replica_n23], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_F95A06AB89972DF5-001/shard-2-001/cores/collection1_shard1_replica_n23/data/]
[junit4] 2> 874187 INFO (qtp1449866319-4198) [n:127.0.0.1:46148_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=46, maxMergeAtOnceExplicit=10, maxMergedSegmentMB=82.958984375, floorSegmentMB=0.96875, forceMergeDeletesPctAllowed=7.808536689504667, segmentsPerTier=16.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7222890085757693
[junit4] 2> 874245 WARN (qtp1449866319-4198) [n:127.0.0.1:46148_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 874627 INFO (qtp1449866319-4198) [n:127.0.0.1:46148_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 874627 INFO (qtp1449866319-4198) [n:127.0.0.1:46148_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 874628 INFO (qtp1449866319-4198) [n:127.0.0.1:46148_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 15000ms;
[junit4] 2> 874628 INFO (qtp1449866319-4198) [n:127.0.0.1:46148_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 874629 INFO (qtp1449866319-4198) [n:127.0.0.1:46148_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=26, maxMergeAtOnceExplicit=50, maxMergedSegmentMB=32.5908203125, floorSegmentMB=0.951171875, forceMergeDeletesPctAllowed=6.617594327471268, segmentsPerTier=7.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 874630 INFO (qtp1449866319-4198) [n:127.0.0.1:46148_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.s.SolrIndexSearcher Opening [Searcher@2bffdf56[collection1_shard1_replica_n23] main]
[junit4] 2> 874631 INFO (qtp1449866319-4198) [n:127.0.0.1:46148_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 874631 INFO (qtp1449866319-4198) [n:127.0.0.1:46148_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 874665 INFO (qtp1449866319-4198) [n:127.0.0.1:46148_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 874681 INFO (searcherExecutor-609-thread-1-processing-n:127.0.0.1:46148_ x:collection1_shard1_replica_n23 s:shard1 c:collection1) [n:127.0.0.1:46148_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.c.SolrCore [collection1_shard1_replica_n23] Registered new searcher Searcher@2bffdf56[collection1_shard1_replica_n23] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 874681 INFO (qtp1449866319-4198) [n:127.0.0.1:46148_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1583254096811393024
[junit4] 2> 874684 INFO (qtp1449866319-4198) [n:127.0.0.1:46148_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.c.ZkController Core needs to recover:collection1_shard1_replica_n23
[junit4] 2> 874715 INFO (qtp1449866319-4198) [n:127.0.0.1:46148_ c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n23&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2305
[junit4] 2> 874729 INFO (updateExecutor-1759-thread-1-processing-n:127.0.0.1:46148_ x:collection1_shard1_replica_n23 s:shard1 c:collection1) [n:127.0.0.1:46148_ c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.u.DefaultSolrCoreState Running recovery
[junit4] 2> 874749 INFO (recoveryExecutor-1760-thread-1-processing-n:127.0.0.1:46148_ x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:46148_ c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
[junit4] 2> 874766 INFO (recoveryExecutor-1760-thread-1-processing-n:127.0.0.1:46148_ x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:46148_ c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
[junit4] 2> 874766 INFO (recoveryExecutor-1760-thread-1-processing-n:127.0.0.1:46148_ x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:46148_ c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1_shard1_replica_n23]
[junit4] 2> 874766 INFO (recoveryExecutor-1760-thread-1-processing-n:127.0.0.1:46148_ x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:46148_ c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 874766 INFO (recoveryExecutor-1760-thread-1-processing-n:127.0.0.1:46148_ x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:46148_ c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.c.RecoveryStrategy Publishing state of core [collection1_shard1_replica_n23] as recovering, leader is [http://127.0.0.1:37078/collection1_shard1_replica_n21/] and I am [http://127.0.0.1:46148/collection1_shard1_replica_n23/]
[junit4] 2> 874775 INFO (qtp74169703-4119) [n:127.0.0.1:34207_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:46148_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=2498
[junit4] 2> 874786 INFO (recoveryExecutor-1760-thread-1-processing-n:127.0.0.1:46148_ x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:46148_ c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:37078]; [WaitForState: action=PREPRECOVERY&core=collection1_shard1_replica_n21&nodeName=127.0.0.1:46148_&coreNodeName=core_node24&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
[junit4] 2> 874792 INFO (OverseerCollectionConfigSetProcessor-98953380010000388-127.0.0.1:34207_-n_0000000000) [n:127.0.0.1:34207_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000006 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 874802 INFO (qtp506102776-4168) [n:127.0.0.1:37078_ ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node24, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true, maxTime: 183 s
[junit4] 2> 874803 INFO (qtp506102776-4168) [n:127.0.0.1:37078_ ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1_shard1_replica_n21, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:46148_, coreNodeName=core_node24, onlyIfActiveCheckResult=false, nodeProps: core_node24:{"core":"collection1_shard1_replica_n23","base_url":"http://127.0.0.1:46148","node_name":"127.0.0.1:46148_","state":"recovering","type":"NRT"}
[junit4] 2> 874803 INFO (qtp506102776-4168) [n:127.0.0.1:37078_ ] o.a.s.h.a.PrepRecoveryOp Waited coreNodeName: core_node24, state: recovering, checkLive: true, onlyIfLeader: true for: 0 seconds.
[junit4] 2> 874803 INFO (qtp506102776-4168) [n:127.0.0.1:37078_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:46148_&onlyIfLeaderActive=true&core=collection1_shard1_replica_n21&coreNodeName=core_node24&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 875326 INFO (recoveryExecutor-1760-thread-1-processing-n:127.0.0.1:46148_ x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:46148_ c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.c.RecoveryStrategy Attempting to PeerSync from [http://127.0.0.1:37078/collection1_shard1_replica_n21/] - recoveringAfterStartup=[true]
[junit4] 2> 875327 INFO (recoveryExecutor-1760-thread-1-processing-n:127.0.0.1:46148_ x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:46148_ c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.u.PeerSync PeerSync: core=collection1_shard1_replica_n23 url=http://127.0.0.1:46148 START replicas=[http://127.0.0.1:37078/collection1_shard1_replica_n21/] nUpdates=100
[junit4] 2> 875437 INFO (qtp506102776-4168) [n:127.0.0.1:37078_ c:collection1 s:shard1 r:core_node22 x:collection1_shard1_replica_n21] o.a.s.u.IndexFingerprint IndexFingerprint millis:86.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
[junit4] 2> 875437 INFO (qtp506102776-4168) [n:127.0.0.1:37078_ c:collection1 s:shard1 r:core_node22 x:collection1_shard1_replica_n21] o.a.s.c.S.Request [collection1_shard1_replica_n21] webapp= path=/get params={distrib=false&qt=/get&getFingerprint=9223372036854775807&wt=javabin&version=2} status=0 QTime=102
[junit4] 2> 875656 INFO (recoveryExecutor-1760-thread-1-processing-n:127.0.0.1:46148_ x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:46148_ c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.u.IndexFingerprint IndexFingerprint millis:218.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
[junit4] 2> 875656 INFO (recoveryExecutor-1760-thread-1-processing-n:127.0.0.1:46148_ x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:46148_ c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.u.PeerSync We are already in sync. No need to do a PeerSync
[junit4] 2> 875656 INFO (recoveryExecutor-1760-thread-1-processing-n:127.0.0.1:46148_ x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:46148_ c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 875656 INFO (recoveryExecutor-1760-thread-1-processing-n:127.0.0.1:46148_ x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:46148_ c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 875656 INFO (recoveryExecutor-1760-thread-1-processing-n:127.0.0.1:46148_ x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:46148_ c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 875676 INFO (recoveryExecutor-1760-thread-1-processing-n:127.0.0.1:46148_ x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:46148_ c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.c.RecoveryStrategy PeerSync stage of recovery was successful.
[junit4] 2> 875676 INFO (recoveryExecutor-1760-thread-1-processing-n:127.0.0.1:46148_ x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:46148_ c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.c.RecoveryStrategy Replaying updates buffered during PeerSync.
[junit4] 2> 875676 INFO (recoveryExecutor-1760-thread-1-processing-n:127.0.0.1:46148_ x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:46148_ c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.c.RecoveryStrategy No replay needed.
[junit4] 2> 875676 INFO (recoveryExecutor-1760-thread-1-processing-n:127.0.0.1:46148_ x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:46148_ c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.c.RecoveryStrategy Registering as Active after recovery.
[junit4] 2> 875899 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_F95A06AB89972DF5-001/shard-3-001 of type NRT
[junit4] 2> 875899 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 876006 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1b39c3a{/,null,AVAILABLE}
[junit4] 2> 876006 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@71b11c40{HTTP/1.1,[http/1.1]}{127.0.0.1:44820}
[junit4] 2> 876006 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.e.j.s.Server Started @888853ms
[junit4] 2> 876006 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_F95A06AB89972DF5-001/tempDir-001/jetty3, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/, hostPort=44820, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_F95A06AB89972DF5-001/shard-3-001/cores}
[junit4] 2> 876006 ERROR (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 876022 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 876022 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 876022 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 876022 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-11-05T19:06:09.815Z
[junit4] 2> 876052 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 876052 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_F95A06AB89972DF5-001/shard-3-001/solr.xml
[junit4] 2> 876234 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 876234 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 876235 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@1adc8034, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 876270 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35937/solr
[junit4] 2> 876383 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [n:127.0.0.1:44820_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 876405 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [n:127.0.0.1:44820_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 876406 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [n:127.0.0.1:44820_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44820_
[junit4] 2> 876407 INFO (zkCallback-1762-thread-1-processing-n:127.0.0.1:46148_) [n:127.0.0.1:46148_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 876407 INFO (zkCallback-1756-thread-1-processing-n:127.0.0.1:37078_) [n:127.0.0.1:37078_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 876408 INFO (zkCallback-1743-thread-3-processing-n:127.0.0.1:34207_) [n:127.0.0.1:34207_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 876441 INFO (zkCallback-1751-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 876441 INFO (zkCallback-1769-thread-1-processing-n:127.0.0.1:44820_) [n:127.0.0.1:44820_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 877075 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [n:127.0.0.1:44820_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1adc8034
[junit4] 2> 877149 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [n:127.0.0.1:44820_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1adc8034
[junit4] 2> 877149 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [n:127.0.0.1:44820_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1adc8034
[junit4] 2> 877151 INFO (TEST-ChaosMonkeySafeLeaderTest.test-seed#[F95A06AB89972DF5]) [n:127.0.0.1:44820_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_F95A06AB89972DF5-001/shard-3-001/cores
[junit4] 2> 877351 INFO (qtp74169703-4121) [n:127.0.0.1:34207_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:44820_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 877454 INFO (OverseerThreadFactory-584-thread-5-processing-n:127.0.0.1:34207_) [n:127.0.0.1:34207_ ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:44820_ for creating new replica
[junit4] 2> 877564 INFO (qtp1963426755-4232) [n:127.0.0.1:44820_ ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n25&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 877564 INFO (qtp1963426755-4232) [n:127.0.0.1:44820_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 878663 INFO (qtp1963426755-4232) [n:127.0.0.1:44820_ c:collection1 s:shard1 x:collection1_shard1_replica_n25] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 878773 INFO (qtp1963426755-4232) [n:127.0.0.1:44820_ c:collection1 s:shard1 x:collection1_shard1_replica_n25] o.a.s.s.IndexSchema [collection1_shard1_replica_n25] Schema name=test
[junit4] 2> 879478 INFO (qtp1963426755-4232) [n:127.0.0.1:44820_ c:collection1 s:shard1 x:collection1_shard1_replica_n25] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 879570 INFO (qtp1963426755-4232) [n:127.0.0.1:44820_ c:collection1 s:shard1 x:collection1_shard1_replica_n25] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n25' using configuration from collection collection1, trusted=true
[junit4] 2> 879570 INFO (qtp1963426755-4232) [n:127.0.0.1:44820_ c:collection1 s:shard1 x:collection1_shard1_replica_n25] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n25' (registry 'solr.core.collection1.shard1.replica_n25') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1adc8034
[junit4] 2> 879570 INFO (qtp1963426755-4232) [n:127.0.0.1:44820_ c:collection1 s:shard1 x:collection1_shard1_replica_n25] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 879570 INFO (qtp1963426755-4232) [n:127.0.0.1:44820_ c:collection1 s:shard1 x:collection1_shard1_replica_n25] o.a.s.c.SolrCore [[collection1_shard1_replica_n25] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_F95A06AB89972DF5-001/shard-3-001/cores/collection1_shard1_replica_n25], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.ChaosMonkeySafeLeaderTest_F95A06AB89972DF5-001/shard-3-001/cores/collection1_shard1_replica_n25/data/]
[junit4] 2> 879614 INFO (qtp1963426755-4232) [n:127.0.0.1:44820_ c:collection1 s:shard1 x:collection1_shard1_replica_n25] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=46, maxMergeAtOnceExplicit=10, maxMergedSegmentMB=82.958984375, floorSegmentMB=0.96875, forceMergeDeletesPctAllowed=7.808536689504667, segmentsPerTier=16.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7222890085757693
[junit4] 2> 879635 WARN (qtp1963426755-4232) [n:127.0.0.1:44820_ c:collection1 s:shard1 x:collection1_shard1_replica_n25] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 879815 INFO (qtp1963426755-4232) [n:127.0.0.1:44820_ c:collection1 s:shard1 x:collection1_shard1_replica_n25] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 879815 INFO (qtp1963426755-4232) [n:127.0.0.1:44820_ c:collection1 s:shard1 x:collection1_shard1_replica_n25] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 879832 INFO (qtp1963426755-4232) [n:127.0.0.1:44820_ c:collection1 s:shard1 x:collection1_shard1_replica_n25] o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 15000ms;
[junit4] 2> 879832 INFO (qtp1963426755-4232) [n:127.0.0.1:44820_ c:collection1 s:shard1 x:collection1_shard1_replica_n25] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 879833 INFO (qtp1963426755-4232) [n:127.0.0.1:44820_ c:collection1 s:shard1 x:collection1_shard1_replica_n25] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=26, maxMergeAtOnceExplicit=50, maxMergedSegmentMB=32.5908203125, floorSegmentMB=0.951171875, forceMergeDeletesPctAllowed=6.617594327471268, segmentsPerTier=7.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 879834 INFO (qtp1963426755-4232) [n:127.0.0.1:44820_ c:collection1 s:shard1 x:collection1_shard1_replica_n25] o.a.s.s.SolrIndexSearcher Opening [Searcher@16ee9b78[collection1_shard1_replica_n25] main]
[junit4] 2> 879836 INFO (qtp1963426755-4232) [n:127.0.0.1:44820_ c:collection1 s:shard1 x:collection1_shard1_replica_n25] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 879836 INFO (qtp1963426755-4232) [n:127.0.0.1:44820_ c:collection1 s:shard1 x:collection1_shard1_replica_n25] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 879869 INFO (qtp1963426755-4232) [n:127.0.0.1:44820_ c:collection1 s:shard1 x:collection1_shard1_replica_n25] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 879871 INFO (searcherExecutor-620-thread-1-processing-n:127.0.0.1:44820_ x:collection1_shard1_replica_n25 s:shard1 c:collection1) [n:127.0.0.1:44820_ c:collection1 s:shard1 x:collection1_shard1_replica_n25] o.a.s.c.SolrCore [collection1_shard1_replica_n25] Registered new searcher Searcher@16ee9b78[collection1_shard1_replica_n25] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 879871 INFO (qtp1963426755-4232) [n:127.0.0.1:44820_ c:collection1 s:shard1 x:collection1_shard1_replica_n25] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1583254102253502464
[junit4] 2> 879907 INFO (qtp1963426755-4232) [n:127.0.0.1:44820_ c:collection1 s:shard1 x:collection1_shard1_replica_n25] o.a.s.c.ZkController Core needs to recover:collection1_shard1_replica_n25
[junit4] 2> 879925 INFO (qtp1963426755-4232) [n:127.0.0.1:44820_ c:collection1 s:shard1 x:collection1_shard1_replica_n25] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n25&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2361
[junit4] 2> 879943 INFO (updateExecutor-1766-thread-1-processing-n:127.0.0.1:44820_ x:collection1_shard1_replica_n25 s:shard1 c:collection1) [n:127.0.0.1:44820_ c:collection1 s:shard1 r:core_node26 x:collection1_shard1_replica_n25] o.a.s.u.DefaultSolrCoreState Running recovery
[junit4] 2> 879944 INFO (recoveryExecutor-1767-thread-1-processing-n:127.0.0.1:44820_ x:collection1_shard1_replica_n25 s:shard1 c:collection1 r:core_node26) [n:127.0.0.1:44820_ c:collection1 s:shard1 r:core_node26 x:collection1_shard1_replica_n25] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
[junit4] 2> 879944 INFO (recoveryExecutor-1767-thread-1-processing-n:127.0.0.1:44820_ x:collection1_shard1_replica_n25 s:shard1 c:collection1 r:core_node26) [n:127.0.0.1:44820_ c:collection1 s:shard1 r:core_node26 x:collection1_shard1_replica_n25] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
[junit4] 2> 879944 INFO (recoveryExecutor-1767-thread-1-processing-n:127.0.0.1:44820_ x:collection1_shard1_replica_n25 s:shard1 c:collection1 r:core_node26) [n:127.0.0.1:44820_ c:collection1 s:shard1 r:core_node26 x:collection1_shard1_replica_n25] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1_shard1_replica_n25]
[junit4] 2> 879944 INFO (recoveryExecutor-1767-thread-1-processing-n:127.0.0.1:44820_ x:collection1_shard1_replica_n25 s:shard1 c:collection1 r:core_node26) [n:127.0.0.1:44820_ c:collection1 s:shard1 r:core_node26 x:collection1_shard1_replica_n25] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 879944 INFO (recoveryExecutor-1767-thread-1-processing-n:127.0.0.1:44820_ x:collection1_shard1_replica_n25 s:shard1 c:collection1 r:core_node26) [n:127.0.0.1:44820_ c:collection1 s:shard1 r:core_node26 x:collection1_shard1_replica_n25] o.a.s.c.RecoveryStrategy Publishing state of
[...truncated too long message...]
inClientExec.java:176)
[junit4] 2> at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
[junit4] 2> at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
[junit4] 2> at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
[junit4] 2> at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
[junit4] 2> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
[junit4] 2> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:542)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
[junit4] 2> at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:183)
[junit4] 2> at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:200)
[junit4] 2> at org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:176)
[junit4] 2> at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:115)
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {rnd_b=PostingsFormat(name=Asserting), a_t=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), id=PostingsFormat(name=Asserting)}, docValues:{_version_=DocValuesFormat(name=Asserting)}, maxPointsInLeafNode=1884, maxMBSortInHeap=7.333583782159135, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@37e139b7), locale=sv, timezone=Pacific/Efate
[junit4] 2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=3,free=250287912,total=527433728
[junit4] 2> NOTE: All tests run in this JVM: [TestSolrCloudWithHadoopAuthPlugin, AtomicUpdatesTest, TestWriterPerf, TestSQLHandler, DistribDocExpirationUpdateProcessorTest, TestQuerySenderNoQuery, NumberUtilsTest, LeaderFailoverAfterPartitionTest, TestCloudPseudoReturnFields, TestRequestStatusCollectionAPI, TestReloadAndDeleteDocs, TestDocumentBuilder, HdfsAutoAddReplicasIntegrationTest, ChaosMonkeySafeLeaderTest]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=ChaosMonkeySafeLeaderTest -Dtests.seed=F95A06AB89972DF5 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=sv -Dtests.timezone=Pacific/Efate -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] 2> REQUEST FAILED for id=1-755
[junit4] 2> java.lang.IllegalStateException: Connection pool shut down
[junit4] 2> at org.apache.http.util.Asserts.check(Asserts.java:34)
[junit4] 2> at org.apache.http.pool.AbstractConnPool.lease(AbstractConnPool.java:189)
[junit4] 2> at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.requestConnection(PoolingHttpClientConnectionManager.java:257)
[junit4] 2> at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:176)
[junit4] 2> at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
[junit4] 2> at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
[junit4] 2> at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
[junit4] 2> at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
[junit4] 2> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
[junit4] 2> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:542)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
[junit4] 2> at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:183)
[junit4] 2> at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:200)
[junit4] 2> at org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:176)
[junit4] 2> at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:115)
[junit4] 2> REQUEST FAILED for id=0-780
[junit4] 2> java.lang.IllegalStateException: Connection pool shut down
[junit4] 2> at org.apache.http.util.Asserts.check(Asserts.java:34)
[junit4] 2> at org.apache.http.pool.AbstractConnPool.lease(AbstractConnPool.java:189)
[junit4] 2> at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.requestConnection(PoolingHttpClientConnectionManager.java:257)
[junit4] 2> at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:176)
[junit4] 2> at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
[junit4] 2> at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
[junit4] 2> at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
[junit4] 2> at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
[junit4] 2> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
[junit4] 2> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:542)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
[junit4] 2> at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:183)
[junit4] 2> at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:200)
[junit4] 2> at org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:176)
[junit4] 2> at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:115)
[junit4] ERROR 0.00s J1 | ChaosMonkeySafeLeaderTest (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest:
[junit4] > 1) Thread[id=4556, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135)
[junit4] > 2) Thread[id=4557, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135)
[junit4] > at __randomizedtesting.SeedInfo.seed([F95A06AB89972DF5]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=4556, name=StoppableIndexingThread, state=BLOCKED, group=TGRP-ChaosMonkeySafeLeaderTest]
[junit4] > at java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006)
[junit4] > at org.apache.http.protocol.BasicHttpContext.setAttribute(BasicHttpContext.java:75)
[junit4] > at org.apache.http.protocol.HttpCoreContext.setAttribute(HttpCoreContext.java:107)
[junit4] > at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:180)
[junit4] > at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
[junit4] > at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
[junit4] > at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
[junit4] > at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
[junit4] > at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:542)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
[junit4] > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:183)
[junit4] > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:200)
[junit4] > at org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:176)
[junit4] > at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:115)
[junit4] > 2) Thread[id=4557, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderTest]
[junit4] > at java.io.PrintStream.println(PrintStream.java:805)
[junit4] > at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:120)
[junit4] > at __randomizedtesting.SeedInfo.seed([F95A06AB89972DF5]:0)
[junit4] Completed [44/749 (1!)] on J1 in 135.97s, 1 test, 1 failure, 2 errors <<< FAILURES!
[...truncated 49437 lines...]
[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1417 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1417/
7 tests failed.
FAILED: org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([CC6B5CC15AF9E770]:0)
FAILED: junit.framework.TestSuite.org.apache.lucene.spatial3d.TestGeo3DPoint
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([CC6B5CC15AF9E770]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest
Error Message:
7 threads leaked from SUITE scope at org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest: 1) Thread[id=21119, name=StoppableCommitThread-SendThread(127.0.0.1:36023), state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1051) 2) Thread[id=21141, name=zkCallback-2800-thread-2, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=21147, name=zkCallback-2800-thread-4, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=21122, name=zkCallback-2800-thread-1, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=21086, name=StoppableCommitThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at java.lang.Thread.sleep(Native Method) at org.apache.solr.cloud.StoppableCommitThread.run(StoppableCommitThread.java:55) 6) Thread[id=21120, name=StoppableCommitThread-EventThread, state=WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501) 7) Thread[id=21146, name=zkCallback-2800-thread-3, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 7 threads leaked from SUITE scope at org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest:
1) Thread[id=21119, name=StoppableCommitThread-SendThread(127.0.0.1:36023), state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1051)
2) Thread[id=21141, name=zkCallback-2800-thread-2, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=21147, name=zkCallback-2800-thread-4, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
4) Thread[id=21122, name=zkCallback-2800-thread-1, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
5) Thread[id=21086, name=StoppableCommitThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.solr.cloud.StoppableCommitThread.run(StoppableCommitThread.java:55)
6) Thread[id=21120, name=StoppableCommitThread-EventThread, state=WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)
7) Thread[id=21146, name=zkCallback-2800-thread-3, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([FAE01138DF2BCB66]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=21119, name=StoppableCommitThread-SendThread(127.0.0.1:36023), state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060) 2) Thread[id=21141, name=zkCallback-2800-thread-2, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=21147, name=zkCallback-2800-thread-4, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=21122, name=zkCallback-2800-thread-1, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=21146, name=zkCallback-2800-thread-3, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=21119, name=StoppableCommitThread-SendThread(127.0.0.1:36023), state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)
2) Thread[id=21141, name=zkCallback-2800-thread-2, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=21147, name=zkCallback-2800-thread-4, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
4) Thread[id=21122, name=zkCallback-2800-thread-1, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
5) Thread[id=21146, name=zkCallback-2800-thread-3, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([FAE01138DF2BCB66]:0)
FAILED: org.apache.solr.client.solrj.impl.CloudSolrClientTest.testRetryUpdatesWhenClusterStateIsStale
Error Message:
Error from server at http://127.0.0.1:40290/solr/stale_state_test_col_shard1_replica_n1: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 404 </title> </head> <body> <h2>HTTP ERROR: 404</h2> <p>Problem accessing /solr/stale_state_test_col_shard1_replica_n1/update. Reason: <pre> Can not find: /solr/stale_state_test_col_shard1_replica_n1/update</pre></p> <hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.20.v20170531</a><hr/> </body> </html>
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:40290/solr/stale_state_test_col_shard1_replica_n1: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 404 </title>
</head>
<body>
<h2>HTTP ERROR: 404</h2>
<p>Problem accessing /solr/stale_state_test_col_shard1_replica_n1/update. Reason:
<pre> Can not find: /solr/stale_state_test_col_shard1_replica_n1/update</pre></p>
<hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.20.v20170531</a><hr/>
</body>
</html>
at __randomizedtesting.SeedInfo.seed([7E0FC110BBF46D56:CA3E59F8581D1B7A]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:607)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:559)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1016)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:883)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:233)
at org.apache.solr.client.solrj.impl.CloudSolrClientTest.testRetryUpdatesWhenClusterStateIsStale(CloudSolrClientTest.java:844)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:110)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest.test
Error Message:
The Monkey ran for over 45 seconds and no jetties were stopped - this is worth investigating!
Stack Trace:
java.lang.AssertionError: The Monkey ran for over 45 seconds and no jetties were stopped - this is worth investigating!
at __randomizedtesting.SeedInfo.seed([FAE01138DF2BCB66:72B42EE271D7A69E]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.ChaosMonkey.stopTheMonkey(ChaosMonkey.java:587)
at org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest.test(ChaosMonkeySafeLeaderWithPullReplicasTest.java:174)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.update.CdcrUpdateLogTest.testSubReader
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([FAE01138DF2BCB66:B7389B5BAFFC1AB4]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.update.CdcrUpdateLogTest.testSubReader(CdcrUpdateLogTest.java:583)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 10757 lines...]
[junit4] Suite: org.apache.lucene.spatial3d.TestGeo3DPoint
[junit4] 2> nov 11, 2017 10:26:54 PM com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
[junit4] 2> WARNING: Suite execution timed out: org.apache.lucene.spatial3d.TestGeo3DPoint
[junit4] 2> 1) Thread[id=9, name=JUnit4-serializer-daemon, state=TIMED_WAITING, group=main]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:50)
[junit4] 2> 2) Thread[id=1, name=main, state=WAITING, group=main]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Thread.join(Thread.java:1252)
[junit4] 2> at java.lang.Thread.join(Thread.java:1326)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:636)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:493)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:251)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:368)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:13)
[junit4] 2> 3) Thread[id=11, name=SUITE-TestGeo3DPoint-seed#[CC6B5CC15AF9E770], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at java.lang.Thread.getStackTrace(Thread.java:1559)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:696)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:693)
[junit4] 2> at java.security.AccessController.doPrivileged(Native Method)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getStackTrace(ThreadLeakControl.java:693)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getThreadsWithTraces(ThreadLeakControl.java:709)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:689)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:65)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:415)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:705)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:139)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:626)
[junit4] 2> 4) Thread[id=12, name=TEST-TestGeo3DPoint.testRandomBig-seed#[CC6B5CC15AF9E770], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoComplexPolygon.isWithin(GeoComplexPolygon.java:366)
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoBaseMembershipShape.isWithin(GeoBaseMembershipShape.java:36)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.verify(TestGeo3DPoint.java:849)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.doTestRandom(TestGeo3DPoint.java:548)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig(TestGeo3DPoint.java:480)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestGeo3DPoint -Dtests.method=testRandomBig -Dtests.seed=CC6B5CC15AF9E770 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=es-CU -Dtests.timezone=Europe/Simferopol -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 7157s J1 | TestGeo3DPoint.testRandomBig <<<
[junit4] > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
[junit4] > at __randomizedtesting.SeedInfo.seed([CC6B5CC15AF9E770]:0)
[junit4] 2> nov 11, 2017 10:26:54 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> nov 11, 2017 10:27:14 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.lucene.spatial3d.TestGeo3DPoint:
[junit4] 2> 1) Thread[id=12, name=TEST-TestGeo3DPoint.testRandomBig-seed#[CC6B5CC15AF9E770], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at org.apache.lucene.spatial3d.geom.Plane.findIntersections(Plane.java:650)
[junit4] 2> at org.apache.lucene.spatial3d.geom.StandardXYZSolid.<init>(StandardXYZSolid.java:188)
[junit4] 2> at org.apache.lucene.spatial3d.geom.XYZSolidFactory.makeXYZSolid(XYZSolidFactory.java:64)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.verify(TestGeo3DPoint.java:847)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.doTestRandom(TestGeo3DPoint.java:548)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig(TestGeo3DPoint.java:480)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> nov 11, 2017 10:27:14 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=12, name=TEST-TestGeo3DPoint.testRandomBig-seed#[CC6B5CC15AF9E770], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> nov 11, 2017 10:27:17 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> SEVERE: There are still zombie threads that couldn't be terminated:
[junit4] 2> 1) Thread[id=12, name=TEST-TestGeo3DPoint.testRandomBig-seed#[CC6B5CC15AF9E770], state=RUNNABLE, group=TGRP-TestGeo3DPoint]
[junit4] 2> at org.apache.lucene.spatial3d.geom.Plane.findIntersections(Plane.java:829)
[junit4] 2> at org.apache.lucene.spatial3d.geom.Plane.recordBounds(Plane.java:1260)
[junit4] 2> at org.apache.lucene.spatial3d.geom.XYZBounds.addPlane(XYZBounds.java:161)
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoStandardPath$PathSegment.getBounds(GeoStandardPath.java:1140)
[junit4] 2> at org.apache.lucene.spatial3d.geom.GeoStandardPath.getBounds(GeoStandardPath.java:427)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.verify(TestGeo3DPoint.java:846)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.doTestRandom(TestGeo3DPoint.java:548)
[junit4] 2> at org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig(TestGeo3DPoint.java:480)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {id=PostingsFormat(name=LuceneFixedGap)}, docValues:{id=DocValuesFormat(name=Direct), point=DocValuesFormat(name=Lucene70)}, maxPointsInLeafNode=1308, maxMBSortInHeap=5.595206355174632, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@32ded155), locale=es-CU, timezone=Europe/Simferopol
[junit4] 2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=2,free=201197968,total=509607936
[junit4] 2> NOTE: All tests run in this JVM: [TestGeo3DPoint]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestGeo3DPoint -Dtests.seed=CC6B5CC15AF9E770 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=es-CU -Dtests.timezone=Europe/Simferopol -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J1 | TestGeo3DPoint (suite) <<<
[junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4] > at __randomizedtesting.SeedInfo.seed([CC6B5CC15AF9E770]:0)
[junit4] Completed [15/15 (1!)] on J1 in 7224.46s, 11 tests, 2 errors <<< FAILURES!
[...truncated 2873 lines...]
[junit4] Suite: org.apache.solr.update.CdcrUpdateLogTest
[junit4] 2> 5066499 INFO (SUITE-CdcrUpdateLogTest-seed#[FAE01138DF2BCB66]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.update.CdcrUpdateLogTest_FAE01138DF2BCB66-001/init-core-data-001
[junit4] 2> 5066500 INFO (SUITE-CdcrUpdateLogTest-seed#[FAE01138DF2BCB66]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 5066501 INFO (SUITE-CdcrUpdateLogTest-seed#[FAE01138DF2BCB66]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 5066501 INFO (SUITE-CdcrUpdateLogTest-seed#[FAE01138DF2BCB66]-worker) [ ] o.a.s.SolrTestCaseJ4 ####initCore
[junit4] 2> 5066501 INFO (SUITE-CdcrUpdateLogTest-seed#[FAE01138DF2BCB66]-worker) [ ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib, /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
[junit4] 2> 5066743 INFO (SUITE-CdcrUpdateLogTest-seed#[FAE01138DF2BCB66]-worker) [ ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 5066875 INFO (SUITE-CdcrUpdateLogTest-seed#[FAE01138DF2BCB66]-worker) [ ] o.a.s.s.IndexSchema [null] Schema name=test
[junit4] 2> 5067695 INFO (SUITE-CdcrUpdateLogTest-seed#[FAE01138DF2BCB66]-worker) [ ] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 5068037 INFO (SUITE-CdcrUpdateLogTest-seed#[FAE01138DF2BCB66]-worker) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@369b4f2d
[junit4] 2> 5068165 INFO (SUITE-CdcrUpdateLogTest-seed#[FAE01138DF2BCB66]-worker) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@369b4f2d
[junit4] 2> 5068165 INFO (SUITE-CdcrUpdateLogTest-seed#[FAE01138DF2BCB66]-worker) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@369b4f2d
[junit4] 2> 5068329 INFO (coreLoadExecutor-2561-thread-1) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 5068366 INFO (coreLoadExecutor-2561-thread-1) [ x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib, /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
[junit4] 2> 5068662 INFO (coreLoadExecutor-2561-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 5068933 INFO (coreLoadExecutor-2561-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 5069373 INFO (coreLoadExecutor-2561-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 5069426 INFO (coreLoadExecutor-2561-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1, trusted=true
[junit4] 2> 5069427 INFO (coreLoadExecutor-2561-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@369b4f2d
[junit4] 2> 5069441 INFO (coreLoadExecutor-2561-thread-1) [ x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 5069441 INFO (coreLoadExecutor-2561-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.update.CdcrUpdateLogTest_FAE01138DF2BCB66-001/init-core-data-001/]
[junit4] 2> 5069461 INFO (coreLoadExecutor-2561-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1021785060, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 5070102 INFO (coreLoadExecutor-2561-thread-1) [ x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog
[junit4] 2> 5070102 INFO (coreLoadExecutor-2561-thread-1) [ x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 5070135 INFO (coreLoadExecutor-2561-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 5070136 INFO (coreLoadExecutor-2561-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 5070136 INFO (coreLoadExecutor-2561-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@4a296fb
[junit4] 2> 5070137 INFO (coreLoadExecutor-2561-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5c81aec4[collection1] main]
[junit4] 2> 5070137 WARN (coreLoadExecutor-2561-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage Cannot write to config directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf; switching to use InMemory storage instead.
[junit4] 2> 5070137 INFO (coreLoadExecutor-2561-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 5070245 INFO (searcherExecutor-2562-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5c81aec4[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 5070245 INFO (coreLoadExecutor-2561-thread-1) [ x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1583805028908400640
[junit4] 2> 5070248 INFO (SUITE-CdcrUpdateLogTest-seed#[FAE01138DF2BCB66]-worker) [ ] o.a.s.SolrTestCaseJ4 ####initCore end
[junit4] 2> 5070266 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testAbsoluteLastVersion
[junit4] 2> 5070267 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
[junit4] 2> 5070268 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.s.SolrIndexSearcher Opening [Searcher@1d2dbf5b[collection1] realtime]
[junit4] 2> 5070268 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={_version_=-9223372036854775807&update.distrib=FROMLEADER&versions=true&wt=json&indent=true}{deleteByQuery=*:* (-9223372036854775807)} 0 1
[junit4] 2> 5070268 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1583805028932517888,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 5070268 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@49620596 commitCommandVersion:1583805028932517888
[junit4] 2> 5070530 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.s.SolrIndexSearcher Opening [Searcher@2bdab490[collection1] main]
[junit4] 2> 5070530 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 5070531 INFO (searcherExecutor-2562-thread-1) [ ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2bdab490[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 5070531 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{commit=} 0 262
[junit4] 2> 5070575 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=614081744
[junit4] 2> 5070575 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 5070575 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@28c45bd6: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@26a9db78
[junit4] 2> 5070632 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 5070632 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4f5157bf: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@23b1a823
[junit4] 2> 5070668 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 5070668 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@28c392a1: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@7fdd6c8e
[junit4] 2> 5070831 INFO (coreCloseExecutor-2567-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@505d965c
[junit4] 2> 5070831 INFO (coreCloseExecutor-2567-thread-1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=1348310620
[junit4] 2> 5070831 INFO (coreCloseExecutor-2567-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2a3e40cd: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@3d6a3788
[junit4] 2> 5070955 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib, /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
[junit4] 2> 5071188 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 5071414 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.s.IndexSchema [null] Schema name=test
[junit4] 2> 5072181 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 5072498 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@369b4f2d
[junit4] 2> 5072571 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@369b4f2d
[junit4] 2> 5072571 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@369b4f2d
[junit4] 2> 5072618 INFO (coreLoadExecutor-2574-thread-1) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 5072635 INFO (coreLoadExecutor-2574-thread-1) [ x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib, /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
[junit4] 2> 5072894 INFO (coreLoadExecutor-2574-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 5072967 INFO (coreLoadExecutor-2574-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 5073706 INFO (coreLoadExecutor-2574-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 5073764 INFO (coreLoadExecutor-2574-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1, trusted=true
[junit4] 2> 5073764 INFO (coreLoadExecutor-2574-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@369b4f2d
[junit4] 2> 5073764 INFO (coreLoadExecutor-2574-thread-1) [ x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 5073764 INFO (coreLoadExecutor-2574-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.update.CdcrUpdateLogTest_FAE01138DF2BCB66-001/init-core-data-001/]
[junit4] 2> 5074192 INFO (coreLoadExecutor-2574-thread-1) [ x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog
[junit4] 2> 5074192 INFO (coreLoadExecutor-2574-thread-1) [ x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 5074192 INFO (coreLoadExecutor-2574-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 5074192 INFO (coreLoadExecutor-2574-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 5074208 INFO (coreLoadExecutor-2574-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1021785060, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 5074208 INFO (coreLoadExecutor-2574-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@41ad8117[collection1] main]
[junit4] 2> 5074209 WARN (coreLoadExecutor-2574-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage Cannot write to config directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf; switching to use InMemory storage instead.
[junit4] 2> 5074209 INFO (coreLoadExecutor-2574-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 5074209 INFO (coreLoadExecutor-2574-thread-1) [ x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1583805033064955904
[junit4] 2> 5074244 INFO (searcherExecutor-2575-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@41ad8117[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 5074422 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={versions=true&wt=json&indent=true}{add=[0 (1583805033113190400)]} 0 167
[junit4] 2> 5074423 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={versions=true&wt=json&indent=true}{add=[1 (1583805033289351168)]} 0 0
[junit4] 2> 5074436 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={versions=true&wt=json&indent=true}{add=[2 (1583805033289351169)]} 0 12
[junit4] 2> 5074436 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={versions=true&wt=json&indent=true}{add=[3 (1583805033302982656)]} 0 0
[junit4] 2> 5074436 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={versions=true&wt=json&indent=true}{add=[4 (1583805033302982657)]} 0 0
[junit4] 2> 5074436 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={versions=true&wt=json&indent=true}{add=[5 (1583805033302982658)]} 0 0
[junit4] 2> 5074436 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={versions=true&wt=json&indent=true}{add=[6 (1583805033302982659)]} 0 0
[junit4] 2> 5074436 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={versions=true&wt=json&indent=true}{add=[7 (1583805033302982660)]} 0 0
[junit4] 2> 5074436 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={versions=true&wt=json&indent=true}{add=[8 (1583805033302982661)]} 0 0
[junit4] 2> 5074436 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={versions=true&wt=json&indent=true}{add=[9 (1583805033302982662)]} 0 0
[junit4] 2> 5074795 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.s.SolrIndexSearcher Opening [Searcher@26e9d935[collection1] realtime]
[junit4] 2> 5074810 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.s.SolrIndexSearcher Opening [Searcher@747ccc0b[collection1] realtime]
[junit4] 2> 5074811 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={versions=true&wt=json&indent=true}{deleteByQuery=*:* (-1583805033304031232)} 0 374
[junit4] 2> 5074811 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1583805033696198656,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 5074811 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@5e3a17bd commitCommandVersion:1583805033696198656
[junit4] 2> 5074920 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.s.SolrIndexSearcher Opening [Searcher@7352c09b[collection1] main]
[junit4] 2> 5074921 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 5074924 INFO (searcherExecutor-2575-thread-1) [ ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@7352c09b[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 5074940 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{commit=} 0 128
[junit4] 2> 5074941 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={versions=true&wt=json&indent=true}{deleteByQuery=*:* (-1583805033831464960)} 0 0
[junit4] 2> 5074941 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={versions=true&wt=json&indent=true}{add=[10 (1583805033832513536)]} 0 0
[junit4] 2> 5074941 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={versions=true&wt=json&indent=true}{add=[11 (1583805033832513537)]} 0 0
[junit4] 2> 5074942 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={versions=true&wt=json&indent=true}{add=[12 (1583805033833562112)]} 0 0
[junit4] 2> 5074942 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={versions=true&wt=json&indent=true}{add=[13 (1583805033833562113)]} 0 0
[junit4] 2> 5074942 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={versions=true&wt=json&indent=true}{add=[14 (1583805033833562114)]} 0 0
[junit4] 2> 5074942 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={versions=true&wt=json&indent=true}{add=[15 (1583805033833562115)]} 0 0
[junit4] 2> 5074942 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={versions=true&wt=json&indent=true}{add=[16 (1583805033833562116)]} 0 0
[junit4] 2> 5074942 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={versions=true&wt=json&indent=true}{add=[17 (1583805033833562117)]} 0 0
[junit4] 2> 5074942 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={versions=true&wt=json&indent=true}{add=[18 (1583805033833562118)]} 0 0
[junit4] 2> 5074942 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={versions=true&wt=json&indent=true}{add=[19 (1583805033833562119)]} 0 0
[junit4] 2> 5074943 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1583805033834610688,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 5074943 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@5e3a17bd commitCommandVersion:1583805033834610688
[junit4] 2> 5075213 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.s.SolrIndexSearcher Opening [Searcher@45fc951a[collection1] main]
[junit4] 2> 5075214 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 5075215 INFO (searcherExecutor-2575-thread-1) [ ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@45fc951a[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_1(8.0.0):c10)))}
[junit4] 2> 5075215 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{commit=} 0 272
[junit4] 2> 5075215 INFO (TEST-CdcrUpdateLogTest.testAbsoluteLastVersion-seed#[FAE01138DF2BCB66]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testAbsoluteLastVersion
[junit4] 2> 5075307 INFO (TEST-CdcrUpdateLogTest.testLogReaderNextOnNewTLog-seed#[FAE01138DF2BCB66]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testLogReaderNextOnNewTLog
[junit4] 2> 5075307 INFO (TEST-CdcrUpdateLogTest.testLogReaderNextOnNewTLog-seed#[FAE01138DF2BCB66]) [ ] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
[junit4] 2> 5075307 INFO (TEST-CdcrUpdateLogTest.testLogReaderNextOnNewTLog-seed#[FAE01138DF2BCB66]) [ ] o.a.s.s.SolrIndexSearcher Opening [Searcher@791a74c6[collection1] realtime]
[junit4] 2> 5075307 INFO (TEST-CdcrUpdateLogTest.testLogReaderNextOnNewTLog-seed#[FAE01138DF2BCB66]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={_version_=-9223372036854775807&update.distrib=FROMLEADER&versions=true&wt=json&indent=true}{deleteByQuery=*:* (-9223372036854775807)} 0 0
[junit4] 2> 5075308 INFO (TEST-CdcrUpdateLogTest.testLogReaderNextOnNewTLog-seed#[FAE01138DF2BCB66]) [ ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1583805034217340928,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 5075308 INFO (TEST-CdcrUpdateLogTest.testLogReaderNextOnNewTLog-seed#[FAE01138DF2BCB66]) [ ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@5e3a17bd commitCommandVersion:1583805034217340928
[junit4] 2> 5075396 INFO (TEST-CdcrUpdateLogTest.testLogReaderNextOnNewTLog-seed#[FAE01138DF2BCB66]) [ ] o.a.s.s.SolrIndexSearcher Opening [Searcher@68db6ba6[collection1] main]
[junit4] 2> 5075396 INFO (TEST-CdcrUpdateLogTest.testLogReaderNextOnNewTLog-seed#[FAE01138DF2BCB66]) [ ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 5075397 INFO (searcherExecutor-2575-thread-1) [ ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@68db6ba6[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 5075397 INFO (TEST-CdcrUpdateLogTest.testLogReaderNextOnNewTLog-seed#[FAE01138DF2BCB66]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{commit=} 0 89
[junit4] 2> 5075397 INFO (TEST-CdcrUpdateLogTest.testLogReaderNextOnNewTLog-seed#[FAE01138DF2BCB66]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1684339788
[junit4] 2> 5075398 INFO (TEST-CdcrUpdateLogTest.testLogReaderNextOnNewTLog-seed#[FAE01138DF2BCB66]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 5075398 INFO (TEST-CdcrUpdateLogTest.testLogReaderNextOnNewTLog-seed#[FAE01138DF2BCB66]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@187327: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@567d175c
[junit4] 2> 5075434 INFO (TEST-CdcrUpdateLogTest.testLogReaderNextOnNewTLog-seed#[FAE01138DF2BCB66]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 5075434 INFO (TEST-CdcrUpdateLogTest.testLogReaderNextOnNewTLog-seed#[FAE01138DF2BCB66]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@648d8cf8: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@23b1a823
[junit4] 2> 5075470 INFO (TEST-CdcrUpdateLogTest.testLogReaderNextOnNewTLog-seed#[FAE01138DF2BCB66]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 5075470 INFO (TEST-CdcrUpdateLogTest.testLogReaderNextOnNewTLog-seed#[FAE01138DF2BCB66]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@298d9e6f: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@7fdd6c8e
[junit4] 2> 5075578 INFO (coreCloseExecutor-2580-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@5c0d704c
[junit4] 2> 5075578 INFO (coreCloseExecutor-2580-thread-1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=1544384588
[junit4] 2> 5075578 INFO (coreCloseExecutor-2580-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@190052a5: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@685e4975
[junit4] 2> 5075672 INFO (TEST-CdcrUpdateLogTest.testLogReaderNextOnNewTLog-seed#[FAE01138DF2BCB66]) [ ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib, /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
[junit4] 2> 5075836 INFO (TEST-CdcrUpdateLogTest.testLogReaderNextOnNewTLog-seed#[FAE01138DF2BCB66]) [ ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 5075911 INFO (TEST-CdcrUpdateLogTest.testLogReaderNextOnNewTLog-seed#[FAE01138DF2BCB66]) [ ] o.a.s.s.IndexSchema [null] Schema name=test
[junit4] 2> 5076515 INFO (TEST-CdcrUpdateLogTest.testLogReaderNextOnNewTLog-seed#[FAE01138DF2BCB66]) [ ] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 5076881 INFO (TEST-CdcrUpdateLogTest.testLogReaderNextOnNewTLog-seed#[FAE01138DF2BCB66]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@369b4f2d
[junit4] 2> 5076900 INFO (TEST-CdcrUpdateLogTest.testLogReaderNextOnNewTLog-seed#[FAE01138DF2BCB66]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@369b4f2d
[junit4] 2> 5076900 INFO (TEST-CdcrUpdateLogTest.testLogReaderNextOnNewTLog-seed#[FAE01138DF2BCB66]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@369b4f2d
[junit4] 2> 5076962 INFO (coreLoadExecutor-2587-thread-1) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 5076963 INFO (coreLoadExecutor-2587-thread-1) [ x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib, /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
[junit4] 2> 5077183 INFO (coreLoadExecutor-2587-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 5077257 INFO (coreLoadExecutor-2587-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 5078060 INFO (coreLoadExecutor-2587-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 5078081 INFO (coreLoadExecutor-2587-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /home/jenkins/jenkins-slave/workspa
[...truncated too long message...]
try=solr.core.stale_state_test_col.shard1.replica_n3, tag=1885025172
[junit4] 2> 247064 INFO (coreCloseExecutor-221-thread-6) [n:127.0.0.1:39351_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1b26044c: rootName = solr_39351, domain = solr.core.stale_state_test_col.shard1.replica_n3, service url = null, agent id = null] for registry solr.core.stale_state_test_col.shard1.replica_n3 / com.codahale.metrics.MetricRegistry@64f8060d
[junit4] 2> 247101 INFO (coreCloseExecutor-219-thread-4) [n:127.0.0.1:41301_solr c:localShardsTestColl s:shard2 r:core_node11 x:localShardsTestColl_shard2_replica_n8] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.localShardsTestColl.shard2.leader, tag=1818199010
[junit4] 2> 247111 INFO (coreCloseExecutor-219-thread-3) [n:127.0.0.1:41301_solr c:2nd_collection s:shard1 r:core_node3 x:2nd_collection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.2nd_collection.shard1.leader, tag=1522485120
[junit4] 2> 247112 INFO (coreCloseExecutor-219-thread-6) [n:127.0.0.1:41301_solr c:localShardsTestColl s:shard3 r:core_node17 x:localShardsTestColl_shard3_replica_n14] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.localShardsTestColl.shard3.leader, tag=1618840894
[junit4] 2> 247122 INFO (coreCloseExecutor-221-thread-6) [n:127.0.0.1:39351_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.stale_state_test_col.shard1.leader, tag=1885025172
[junit4] 2> 247320 INFO (coreCloseExecutor-220-thread-7) [n:127.0.0.1:40290_solr c:localShardsTestColl s:shard2 r:core_node13 x:localShardsTestColl_shard2_replica_n10] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.localShardsTestColl.shard2.leader, tag=741900051
[junit4] 2> 247320 INFO (coreCloseExecutor-220-thread-8) [n:127.0.0.1:40290_solr c:foo s:shard1 r:core_node5 x:foo_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.foo.shard1.replica_n2, tag=404734228
[junit4] 2> 247320 INFO (coreCloseExecutor-220-thread-8) [n:127.0.0.1:40290_solr c:foo s:shard1 r:core_node5 x:foo_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@739233b: rootName = solr_40290, domain = solr.core.foo.shard1.replica_n2, service url = null, agent id = null] for registry solr.core.foo.shard1.replica_n2 / com.codahale.metrics.MetricRegistry@7c91139b
[junit4] 2> 247413 INFO (coreCloseExecutor-220-thread-8) [n:127.0.0.1:40290_solr c:foo s:shard1 r:core_node5 x:foo_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.foo.shard1.leader, tag=404734228
[junit4] 2> 247413 INFO (coreCloseExecutor-220-thread-9) [n:127.0.0.1:40290_solr c:overwrite s:shard1 r:core_node2 x:overwrite_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.overwrite.shard1.replica_n1, tag=1812867044
[junit4] 2> 247413 INFO (coreCloseExecutor-220-thread-9) [n:127.0.0.1:40290_solr c:overwrite s:shard1 r:core_node2 x:overwrite_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@285d9cc0: rootName = solr_40290, domain = solr.core.overwrite.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.overwrite.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@4af0c080
[junit4] 2> 247639 INFO (coreCloseExecutor-220-thread-9) [n:127.0.0.1:40290_solr c:overwrite s:shard1 r:core_node2 x:overwrite_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.overwrite.shard1.leader, tag=1812867044
[junit4] 2> 247649 INFO (coreCloseExecutor-220-thread-2) [n:127.0.0.1:40290_solr c:multicollection1 s:shard2 r:core_node4 x:multicollection1_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.multicollection1.shard2.leader, tag=501428183
[junit4] 2> 247649 INFO (coreCloseExecutor-220-thread-1) [n:127.0.0.1:40290_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard2.leader, tag=263522253
[junit4] 2> 247649 INFO (coreCloseExecutor-220-thread-1) [n:127.0.0.1:40290_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter close.
[junit4] 2> 247649 INFO (coreCloseExecutor-220-thread-1) [n:127.0.0.1:40290_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1978b268 commitCommandVersion:0
[junit4] 2> 247649 INFO (coreCloseExecutor-220-thread-3) [n:127.0.0.1:40290_solr c:multicollection2 s:shard1 r:core_node3 x:multicollection2_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.multicollection2.shard1.leader, tag=1650587836
[junit4] 2> 247665 INFO (jetty-closer-42-thread-2) [ ] o.a.s.c.Overseer Overseer (id=98988053482635271-127.0.0.1:39351_solr-n_0000000000) closing
[junit4] 2> 247668 INFO (OverseerStateUpdate-98988053482635271-127.0.0.1:39351_solr-n_0000000000) [n:127.0.0.1:39351_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:39351_solr
[junit4] 2> 247723 WARN (OverseerAutoScalingTriggerThread-98988053482635271-127.0.0.1:39351_solr-n_0000000000) [n:127.0.0.1:39351_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 248109 INFO (jetty-closer-42-thread-3) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@51d74340{/solr,null,UNAVAILABLE}
[junit4] 2> 249558 INFO (jetty-closer-42-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@4e9abef5{/solr,null,UNAVAILABLE}
[junit4] 2> 249643 INFO (jetty-closer-42-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@c97bfc4{/solr,null,UNAVAILABLE}
[junit4] 2> 249687 ERROR (SUITE-CloudSolrClientTest-seed#[7E0FC110BBF46D56]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 249688 INFO (SUITE-CloudSolrClientTest-seed#[7E0FC110BBF46D56]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:35576 35576
[junit4] 2> 249817 INFO (Thread-55) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:35576 35576
[junit4] 2> 249850 WARN (Thread-55) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 8 /solr/clusterprops.json
[junit4] 2> 8 /solr/aliases.json
[junit4] 2> 3 /solr/security.json
[junit4] 2> 3 /solr/configs/conf
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 42 /solr/collections/localShardsTestColl/state.json
[junit4] 2> 30 /solr/collections/foo/state.json
[junit4] 2> 18 /solr/collections/2nd_collection/state.json
[junit4] 2> 18 /solr/collections/multicollection2/state.json
[junit4] 2> 16 /solr/collections/stale_state_test_col/state.json
[junit4] 2> 16 /solr/collections/collection1/state.json
[junit4] 2> 14 /solr/collections/nemesis/state.json
[junit4] 2> 14 /solr/collections/multicollection1/state.json
[junit4] 2> 8 /solr/clusterstate.json
[junit4] 2> 6 /solr/collections/overwrite/state.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 8 /solr/live_nodes
[junit4] 2> 8 /solr/collections
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.impl.CloudSolrClientTest_7E0FC110BBF46D56-001
[junit4] 2> Nov 11, 2017 10:05:54 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {_version_=PostingsFormat(name=Memory), multiDefault=PostingsFormat(name=LuceneVarGapDocFreqInterval), a_t=PostingsFormat(name=Memory), title_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), intDefault=PostingsFormat(name=Memory), id=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), text=FST50, timestamp=PostingsFormat(name=Memory)}, docValues:{_version_=DocValuesFormat(name=Asserting), multiDefault=DocValuesFormat(name=Memory), title_s=DocValuesFormat(name=Memory), intDefault=DocValuesFormat(name=Asserting), id=DocValuesFormat(name=Lucene70), timestamp=DocValuesFormat(name=Asserting)}, maxPointsInLeafNode=1079, maxMBSortInHeap=7.3880423248818365, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@46147ca9), locale=ms, timezone=America/Detroit
[junit4] 2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=1,free=145684280,total=385875968
[junit4] 2> NOTE: All tests run in this JVM: [GetByIdTest, JdbcDriverTest, SelectWithEvaluatorsTest, DivideEvaluatorTest, SolrParamTest, SolrExampleXMLTest, ExclusiveOrEvaluatorTest, CommonParamsTest, CloudSolrClientTest]
[junit4] Completed [51/146 (1!)] on J2 in 127.59s, 16 tests, 1 error <<< FAILURES!
[...truncated 46586 lines...]
[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1416 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1416/
5 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest
Error Message:
2 threads leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest: 1) Thread[id=2249, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-HdfsChaosMonkeySafeLeaderTest] at java.lang.Thread.sleep(Native Method) at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135) 2) Thread[id=2248, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-HdfsChaosMonkeySafeLeaderTest] at java.lang.Thread.sleep(Native Method) at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest:
1) Thread[id=2249, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-HdfsChaosMonkeySafeLeaderTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135)
2) Thread[id=2248, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-HdfsChaosMonkeySafeLeaderTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135)
at __randomizedtesting.SeedInfo.seed([B5660C1FA2E71586]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=2249, name=StoppableIndexingThread, state=BLOCKED, group=TGRP-HdfsChaosMonkeySafeLeaderTest] at java.security.AccessController.doPrivileged(Native Method) at com.carrotsearch.ant.tasks.junit4.events.Serializer.flushQueue(Serializer.java:114) at com.carrotsearch.ant.tasks.junit4.events.Serializer.serialize(Serializer.java:99) at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain$3$2.write(SlaveMain.java:472) at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) at java.io.PrintStream.flush(PrintStream.java:338) at java.io.FilterOutputStream.flush(FilterOutputStream.java:140) at java.io.PrintStream.write(PrintStream.java:482) at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221) at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291) at sun.nio.cs.StreamEncoder.flushBuffer(StreamEncoder.java:104) at java.io.OutputStreamWriter.flushBuffer(OutputStreamWriter.java:185) at java.io.PrintStream.write(PrintStream.java:527) at java.io.PrintStream.print(PrintStream.java:669) at java.io.PrintStream.println(PrintStream.java:823) at java.lang.Throwable$WrappedPrintStream.println(Throwable.java:748) at java.lang.Throwable.printStackTrace(Throwable.java:658) at java.lang.Throwable.printStackTrace(Throwable.java:643) at java.lang.Throwable.printStackTrace(Throwable.java:634) at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:121) 2) Thread[id=2248, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-HdfsChaosMonkeySafeLeaderTest] at java.util.HashMap.putVal(HashMap.java:629) at java.util.HashMap.put(HashMap.java:612) at org.apache.solr.common.params.ModifiableSolrParams.set(ModifiableSolrParams.java:93) at org.apache.solr.client.solrj.request.AbstractUpdateRequest.setParam(AbstractUpdateRequest.java:101) at org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:175) at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:115)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=2249, name=StoppableIndexingThread, state=BLOCKED, group=TGRP-HdfsChaosMonkeySafeLeaderTest]
at java.security.AccessController.doPrivileged(Native Method)
at com.carrotsearch.ant.tasks.junit4.events.Serializer.flushQueue(Serializer.java:114)
at com.carrotsearch.ant.tasks.junit4.events.Serializer.serialize(Serializer.java:99)
at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain$3$2.write(SlaveMain.java:472)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
at java.io.PrintStream.flush(PrintStream.java:338)
at java.io.FilterOutputStream.flush(FilterOutputStream.java:140)
at java.io.PrintStream.write(PrintStream.java:482)
at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
at sun.nio.cs.StreamEncoder.flushBuffer(StreamEncoder.java:104)
at java.io.OutputStreamWriter.flushBuffer(OutputStreamWriter.java:185)
at java.io.PrintStream.write(PrintStream.java:527)
at java.io.PrintStream.print(PrintStream.java:669)
at java.io.PrintStream.println(PrintStream.java:823)
at java.lang.Throwable$WrappedPrintStream.println(Throwable.java:748)
at java.lang.Throwable.printStackTrace(Throwable.java:658)
at java.lang.Throwable.printStackTrace(Throwable.java:643)
at java.lang.Throwable.printStackTrace(Throwable.java:634)
at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:121)
2) Thread[id=2248, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-HdfsChaosMonkeySafeLeaderTest]
at java.util.HashMap.putVal(HashMap.java:629)
at java.util.HashMap.put(HashMap.java:612)
at org.apache.solr.common.params.ModifiableSolrParams.set(ModifiableSolrParams.java:93)
at org.apache.solr.client.solrj.request.AbstractUpdateRequest.setParam(AbstractUpdateRequest.java:101)
at org.apache.solr.cloud.StoppableIndexingThread.indexDocs(StoppableIndexingThread.java:175)
at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:115)
at __randomizedtesting.SeedInfo.seed([B5660C1FA2E71586]:0)
FAILED: org.apache.solr.cloud.MultiThreadedOCPTest.test
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([B5660C1FA2E71586:3D3233C50C1B787E]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.cloud.MultiThreadedOCPTest.testFillWorkQueue(MultiThreadedOCPTest.java:112)
at org.apache.solr.cloud.MultiThreadedOCPTest.test(MultiThreadedOCPTest.java:67)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest.test
Error Message:
The Monkey ran for over 45 seconds and no jetties were stopped - this is worth investigating!
Stack Trace:
java.lang.AssertionError: The Monkey ran for over 45 seconds and no jetties were stopped - this is worth investigating!
at __randomizedtesting.SeedInfo.seed([B5660C1FA2E71586:3D3233C50C1B787E]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.ChaosMonkey.stopTheMonkey(ChaosMonkey.java:587)
at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.test(ChaosMonkeySafeLeaderTest.java:133)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.update.CdcrUpdateLogTest.testSubReader
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([B5660C1FA2E71586:F8BE867CD230C454]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.update.CdcrUpdateLogTest.testSubReader(CdcrUpdateLogTest.java:583)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12847 lines...]
[junit4] Suite: org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_B5660C1FA2E71586-001/init-core-data-001
[junit4] 2> 716675 WARN (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[B5660C1FA2E71586]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=2 numCloses=2
[junit4] 2> 716695 INFO (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[B5660C1FA2E71586]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 716697 INFO (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[B5660C1FA2E71586]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 716697 INFO (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[B5660C1FA2E71586]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 716697 INFO (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[B5660C1FA2E71586]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 2> 722627 WARN (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[B5660C1FA2E71586]-worker) [ ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 730822 WARN (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[B5660C1FA2E71586]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 732567 INFO (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[B5660C1FA2E71586]-worker) [ ] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
[junit4] 2> 732730 WARN (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[B5660C1FA2E71586]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 735684 INFO (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[B5660C1FA2E71586]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 736114 INFO (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[B5660C1FA2E71586]-worker) [ ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/hdfs to ./temp/Jetty_lucene2.us.west_apache_org_34528_hdfs____.a6r6fw/webapp
[junit4] 2> 739044 INFO (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[B5660C1FA2E71586]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@lucene2-us-west.apache.org:34528
[junit4] 2> 747303 WARN (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[B5660C1FA2E71586]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 747343 INFO (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[B5660C1FA2E71586]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 747476 INFO (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[B5660C1FA2E71586]-worker) [ ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_35850_datanode____z5vc7s/webapp
[junit4] 2> 750162 INFO (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[B5660C1FA2E71586]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35850
[junit4] 2> 754458 WARN (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[B5660C1FA2E71586]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 754475 INFO (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[B5660C1FA2E71586]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 754692 INFO (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[B5660C1FA2E71586]-worker) [ ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_42335_datanode____lpd9mm/webapp
[junit4] 2> 757838 INFO (SUITE-HdfsChaosMonkeySafeLeaderTest-seed#[B5660C1FA2E71586]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42335
[junit4] 2> 762543 ERROR (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_B5660C1FA2E71586-001/tempDir-001/hdfsBaseDir/data/data3/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_B5660C1FA2E71586-001/tempDir-001/hdfsBaseDir/data/data4/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:44071) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 762635 ERROR (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_B5660C1FA2E71586-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_B5660C1FA2E71586-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:44071) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 764588 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x24b278af195fe0: from storage DS-fe3bf6de-44cb-4c0a-860c-d3dc1b75fcf1 node DatanodeRegistration(127.0.0.1:34876, datanodeUuid=ce1fb206-a557-4548-b3be-0ed4896a31fa, infoPort=36828, infoSecurePort=0, ipcPort=37654, storageInfo=lv=-56;cid=testClusterID;nsid=196133132;c=0), blocks: 0, hasStaleStorage: true, processing time: 2 msecs
[junit4] 2> 764606 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x24b278b02eb115: from storage DS-5b1ef6fc-b102-4528-bfb9-dccb02c69102 node DatanodeRegistration(127.0.0.1:34176, datanodeUuid=5100abb3-33d2-44ec-aa35-ca1f4af06491, infoPort=36766, infoSecurePort=0, ipcPort=39602, storageInfo=lv=-56;cid=testClusterID;nsid=196133132;c=0), blocks: 0, hasStaleStorage: true, processing time: 17 msecs
[junit4] 2> 764606 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x24b278af195fe0: from storage DS-88db529a-4630-4e16-923a-67e651bb0728 node DatanodeRegistration(127.0.0.1:34876, datanodeUuid=ce1fb206-a557-4548-b3be-0ed4896a31fa, infoPort=36828, infoSecurePort=0, ipcPort=37654, storageInfo=lv=-56;cid=testClusterID;nsid=196133132;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 764637 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x24b278b02eb115: from storage DS-164f9060-5e04-4e55-9080-a6848d522ac1 node DatanodeRegistration(127.0.0.1:34176, datanodeUuid=5100abb3-33d2-44ec-aa35-ca1f4af06491, infoPort=36766, infoSecurePort=0, ipcPort=39602, storageInfo=lv=-56;cid=testClusterID;nsid=196133132;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 768131 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 768131 INFO (Thread-495) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 768131 INFO (Thread-495) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 768132 ERROR (Thread-495) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 768239 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.c.ZkTestServer start zk server on port:33407
[junit4] 2> 768517 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 768518 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 768519 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 768536 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 768536 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 768537 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 768538 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 768538 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 768539 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 768552 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 768568 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 768641 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
[junit4] 2> 769703 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 770236 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@32faace3{/,null,AVAILABLE}
[junit4] 2> 770236 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@5ab4b877{HTTP/1.1,[http/1.1]}{127.0.0.1:38059}
[junit4] 2> 770236 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.e.j.s.Server Started @782976ms
[junit4] 2> 770236 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://lucene2-us-west.apache.org:44071/hdfs__lucene2-us-west.apache.org_44071__home_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-master_checkout_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_B5660C1FA2E71586-001_tempDir-002_control_data, hostContext=/, hostPort=38059, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_B5660C1FA2E71586-001/control-001/cores}
[junit4] 2> 770236 ERROR (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 770236 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 770237 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 770237 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 770237 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-11-09T09:33:47.059Z
[junit4] 2> 770382 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 770382 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_B5660C1FA2E71586-001/control-001/solr.xml
[junit4] 2> 770453 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 770453 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 770525 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@58af9533, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 770542 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33407/solr
[junit4] 2> 771482 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [n:127.0.0.1:38059_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 771483 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [n:127.0.0.1:38059_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:38059_
[junit4] 2> 771483 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [n:127.0.0.1:38059_ ] o.a.s.c.Overseer Overseer (id=98973779555385348-127.0.0.1:38059_-n_0000000000) starting
[junit4] 2> 771721 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [n:127.0.0.1:38059_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38059_
[junit4] 2> 771737 INFO (zkCallback-267-thread-1-processing-n:127.0.0.1:38059_) [n:127.0.0.1:38059_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 772198 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [n:127.0.0.1:38059_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58af9533
[junit4] 2> 772252 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [n:127.0.0.1:38059_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58af9533
[junit4] 2> 772253 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [n:127.0.0.1:38059_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58af9533
[junit4] 2> 772266 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [n:127.0.0.1:38059_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_B5660C1FA2E71586-001/control-001/cores
[junit4] 2> 772774 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 772847 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33407/solr ready
[junit4] 2> 773019 INFO (qtp418682886-1844) [n:127.0.0.1:38059_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:38059_&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 773315 INFO (OverseerThreadFactory-680-thread-1-processing-n:127.0.0.1:38059_) [n:127.0.0.1:38059_ ] o.a.s.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 773528 INFO (qtp418682886-1846) [n:127.0.0.1:38059_ ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 773528 INFO (qtp418682886-1846) [n:127.0.0.1:38059_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 773691 INFO (zkCallback-267-thread-1-processing-n:127.0.0.1:38059_) [n:127.0.0.1:38059_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 774789 INFO (qtp418682886-1846) [n:127.0.0.1:38059_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 774900 INFO (qtp418682886-1846) [n:127.0.0.1:38059_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
[junit4] 2> 779894 WARN (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]-SendThread(127.0.0.1:33407)) [ ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 7046ms for sessionid 0x15fa020cc7a0005
[junit4] 2> 779894 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15fa020cc7a0005, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 780033 WARN (zkCallback-272-thread-3) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@77cf76aa name: ZooKeeperConnection Watcher:127.0.0.1:33407/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
[junit4] 2> 780033 WARN (zkCallback-272-thread-3) [ ] o.a.s.c.c.ConnectionManager zkClient has disconnected
[junit4] 2> 780546 INFO (qtp418682886-1846) [n:127.0.0.1:38059_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 780601 INFO (qtp418682886-1846) [n:127.0.0.1:38059_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
[junit4] 2> 780619 INFO (qtp418682886-1846) [n:127.0.0.1:38059_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58af9533
[junit4] 2> 780819 INFO (qtp418682886-1846) [n:127.0.0.1:38059_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://lucene2-us-west.apache.org:44071/solr_hdfs_home
[junit4] 2> 780819 INFO (qtp418682886-1846) [n:127.0.0.1:38059_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 780819 INFO (qtp418682886-1846) [n:127.0.0.1:38059_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 780819 INFO (qtp418682886-1846) [n:127.0.0.1:38059_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_B5660C1FA2E71586-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[hdfs://lucene2-us-west.apache.org:44071/solr_hdfs_home/control_collection/core_node2/data/]
[junit4] 2> 780821 INFO (qtp418682886-1846) [n:127.0.0.1:38059_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:44071/solr_hdfs_home/control_collection/core_node2/data/snapshot_metadata
[junit4] 2> 781317 INFO (qtp418682886-1846) [n:127.0.0.1:38059_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 781387 INFO (qtp418682886-1846) [n:127.0.0.1:38059_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 781387 INFO (qtp418682886-1846) [n:127.0.0.1:38059_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 787443 INFO (qtp418682886-1846) [n:127.0.0.1:38059_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 787462 INFO (qtp418682886-1846) [n:127.0.0.1:38059_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:44071/solr_hdfs_home/control_collection/core_node2/data
[junit4] 2> 787809 INFO (qtp418682886-1846) [n:127.0.0.1:38059_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:44071/solr_hdfs_home/control_collection/core_node2/data/index
[junit4] 2> 788119 INFO (qtp418682886-1846) [n:127.0.0.1:38059_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 788119 INFO (qtp418682886-1846) [n:127.0.0.1:38059_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 788119 INFO (qtp418682886-1846) [n:127.0.0.1:38059_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 788449 INFO (qtp418682886-1846) [n:127.0.0.1:38059_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 788449 INFO (qtp418682886-1846) [n:127.0.0.1:38059_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@257c0df3
[junit4] 2> 790853 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:34176 is added to blk_1073741825_1001{UCState=COMMITTED, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-164f9060-5e04-4e55-9080-a6848d522ac1:NORMAL:127.0.0.1:34176|RBW], ReplicaUC[[DISK]DS-88db529a-4630-4e16-923a-67e651bb0728:NORMAL:127.0.0.1:34876|RBW]]} size 69
[junit4] 2> 790854 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:34876 is added to blk_1073741825_1001 size 69
[junit4] 2> 791318 WARN (qtp418682886-1846) [n:127.0.0.1:38059_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 792124 INFO (qtp418682886-1846) [n:127.0.0.1:38059_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 792124 INFO (qtp418682886-1846) [n:127.0.0.1:38059_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 792124 INFO (qtp418682886-1846) [n:127.0.0.1:38059_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 792432 INFO (qtp418682886-1846) [n:127.0.0.1:38059_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 15000ms;
[junit4] 2> 792432 INFO (qtp418682886-1846) [n:127.0.0.1:38059_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 792524 INFO (qtp418682886-1846) [n:127.0.0.1:38059_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=37, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=1.8720703125, noCFSRatio=0.0]
[junit4] 2> 793392 INFO (qtp418682886-1846) [n:127.0.0.1:38059_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3fdda0eb[control_collection_shard1_replica_n1] main]
[junit4] 2> 793442 INFO (qtp418682886-1846) [n:127.0.0.1:38059_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 793463 INFO (qtp418682886-1846) [n:127.0.0.1:38059_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 793466 INFO (qtp418682886-1846) [n:127.0.0.1:38059_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 793517 INFO (searcherExecutor-683-thread-1-processing-n:127.0.0.1:38059_ x:control_collection_shard1_replica_n1 s:shard1 c:control_collection) [n:127.0.0.1:38059_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@3fdda0eb[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 793519 INFO (qtp418682886-1846) [n:127.0.0.1:38059_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1583580499506364416
[junit4] 2> 793663 INFO (qtp418682886-1846) [n:127.0.0.1:38059_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 793663 INFO (qtp418682886-1846) [n:127.0.0.1:38059_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 793663 INFO (qtp418682886-1846) [n:127.0.0.1:38059_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:38059/control_collection_shard1_replica_n1/
[junit4] 2> 793663 INFO (qtp418682886-1846) [n:127.0.0.1:38059_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 793663 INFO (qtp418682886-1846) [n:127.0.0.1:38059_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:38059/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 793663 INFO (qtp418682886-1846) [n:127.0.0.1:38059_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 793700 INFO (qtp418682886-1846) [n:127.0.0.1:38059_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:38059/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 793738 INFO (zkCallback-267-thread-2-processing-n:127.0.0.1:38059_) [n:127.0.0.1:38059_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 793773 INFO (qtp418682886-1846) [n:127.0.0.1:38059_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 793793 INFO (qtp418682886-1846) [n:127.0.0.1:38059_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=20265
[junit4] 2> 793911 INFO (qtp418682886-1844) [n:127.0.0.1:38059_ ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 793955 INFO (zkCallback-267-thread-2-processing-n:127.0.0.1:38059_) [n:127.0.0.1:38059_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 794196 INFO (OverseerCollectionConfigSetProcessor-98973779555385348-127.0.0.1:38059_-n_0000000000) [n:127.0.0.1:38059_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 794915 INFO (qtp418682886-1844) [n:127.0.0.1:38059_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:38059_&wt=javabin&version=2} status=0 QTime=21895
[junit4] 2> 795070 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 795071 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33407/solr ready
[junit4] 2> 795071 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 795072 INFO (qtp418682886-1843) [n:127.0.0.1:38059_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=3&createNodeSet=&stateFormat=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 795105 INFO (OverseerThreadFactory-680-thread-2-processing-n:127.0.0.1:38059_) [n:127.0.0.1:38059_ ] o.a.s.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 795106 WARN (OverseerThreadFactory-680-thread-2-processing-n:127.0.0.1:38059_) [n:127.0.0.1:38059_ ] o.a.s.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
[junit4] 2> 795328 INFO (qtp418682886-1843) [n:127.0.0.1:38059_ ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 795328 INFO (qtp418682886-1843) [n:127.0.0.1:38059_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=3&createNodeSet=&stateFormat=2&wt=javabin&version=2} status=0 QTime=256
[junit4] 2> 797119 INFO (OverseerCollectionConfigSetProcessor-98973779555385348-127.0.0.1:38059_-n_0000000000) [n:127.0.0.1:38059_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 797188 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_B5660C1FA2E71586-001/shard-1-001 of type NRT
[junit4] 2> 797189 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 797331 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1028eeff{/,null,AVAILABLE}
[junit4] 2> 797368 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@e3a09f4{HTTP/1.1,[http/1.1]}{127.0.0.1:34026}
[junit4] 2> 797368 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.e.j.s.Server Started @810107ms
[junit4] 2> 797372 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://lucene2-us-west.apache.org:44071/hdfs__lucene2-us-west.apache.org_44071__home_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-master_checkout_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_B5660C1FA2E71586-001_tempDir-002_jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=34026, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_B5660C1FA2E71586-001/shard-1-001/cores}
[junit4] 2> 797372 ERROR (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 797372 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 797372 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 797372 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 797372 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-11-09T09:34:14.194Z
[junit4] 2> 797498 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 797498 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_B5660C1FA2E71586-001/shard-1-001/solr.xml
[junit4] 2> 797535 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 797535 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 797536 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@58af9533, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 797571 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33407/solr
[junit4] 2> 797757 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [n:127.0.0.1:34026_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 797772 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [n:127.0.0.1:34026_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 797773 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [n:127.0.0.1:34026_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34026_
[junit4] 2> 797880 INFO (zkCallback-275-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 797909 INFO (zkCallback-267-thread-1-processing-n:127.0.0.1:38059_) [n:127.0.0.1:38059_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 797953 INFO (zkCallback-280-thread-1-processing-n:127.0.0.1:34026_) [n:127.0.0.1:34026_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 798356 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [n:127.0.0.1:34026_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58af9533
[junit4] 2> 798466 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [n:127.0.0.1:34026_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58af9533
[junit4] 2> 798466 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [n:127.0.0.1:34026_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58af9533
[junit4] 2> 798468 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [n:127.0.0.1:34026_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_B5660C1FA2E71586-001/shard-1-001/cores
[junit4] 2> 799034 INFO (qtp2102923143-1905) [n:127.0.0.1:34026_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:34026_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 799052 INFO (OverseerThreadFactory-680-thread-3-processing-n:127.0.0.1:38059_) [n:127.0.0.1:38059_ ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:34026_ for creating new replica
[junit4] 2> 799144 INFO (qtp2102923143-1902) [n:127.0.0.1:34026_ ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n61&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 799144 INFO (qtp2102923143-1902) [n:127.0.0.1:34026_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 799438 INFO (qtp2102923143-1902) [n:127.0.0.1:34026_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 799785 INFO (qtp2102923143-1902) [n:127.0.0.1:34026_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.s.IndexSchema [collection1_shard2_replica_n61] Schema name=test
[junit4] 2> 800837 INFO (qtp2102923143-1902) [n:127.0.0.1:34026_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 801054 INFO (qtp2102923143-1902) [n:127.0.0.1:34026_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n61' using configuration from collection collection1, trusted=true
[junit4] 2> 801054 INFO (qtp2102923143-1902) [n:127.0.0.1:34026_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_n61' (registry 'solr.core.collection1.shard2.replica_n61') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58af9533
[junit4] 2> 801054 INFO (qtp2102923143-1902) [n:127.0.0.1:34026_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://lucene2-us-west.apache.org:44071/solr_hdfs_home
[junit4] 2> 801093 INFO (qtp2102923143-1902) [n:127.0.0.1:34026_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 801093 INFO (qtp2102923143-1902) [n:127.0.0.1:34026_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 801093 INFO (qtp2102923143-1902) [n:127.0.0.1:34026_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.SolrCore [[collection1_shard2_replica_n61] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_B5660C1FA2E71586-001/shard-1-001/cores/collection1_shard2_replica_n61], dataDir=[hdfs://lucene2-us-west.apache.org:44071/solr_hdfs_home/collection1/core_node62/data/]
[junit4] 2> 801095 INFO (qtp2102923143-1902) [n:127.0.0.1:34026_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:44071/solr_hdfs_home/collection1/core_node62/data/snapshot_metadata
[junit4] 2> 801260 INFO (qtp2102923143-1902) [n:127.0.0.1:34026_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 801260 INFO (qtp2102923143-1902) [n:127.0.0.1:34026_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 801260 INFO (qtp2102923143-1902) [n:127.0.0.1:34026_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 801606 INFO (qtp2102923143-1902) [n:127.0.0.1:34026_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 801625 INFO (qtp2102923143-1902) [n:127.0.0.1:34026_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:44071/solr_hdfs_home/collection1/core_node62/data
[junit4] 2> 801815 INFO (qtp2102923143-1902) [n:127.0.0.1:34026_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:44071/solr_hdfs_home/collection1/core_node62/data/index
[junit4] 2> 801961 INFO (qtp2102923143-1902) [n:127.0.0.1:34026_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 801961 INFO (qtp2102923143-1902) [n:127.0.0.1:34026_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 801961 INFO (qtp2102923143-1902) [n:127.0.0.1:34026_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 802169 INFO (qtp2102923143-1902) [n:127.0.0.1:34026_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 802169 INFO (qtp2102923143-1902) [n:127.0.0.1:34026_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@679f8045
[junit4] 2> 802636 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:34876 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-164f9060-5e04-4e55-9080-a6848d522ac1:NORMAL:127.0.0.1:34176|RBW], ReplicaUC[[DISK]DS-88db529a-4630-4e16-923a-67e651bb0728:NORMAL:127.0.0.1:34876|FINALIZED]]} size 0
[junit4] 2> 802672 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:34176 is added to blk_1073741826_1002 size 69
[junit4] 2> 802921 WARN (qtp2102923143-1902) [n:127.0.0.1:34026_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 803415 INFO (qtp2102923143-1902) [n:127.0.0.1:34026_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 803415 INFO (qtp2102923143-1902) [n:127.0.0.1:34026_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 803415 INFO (qtp2102923143-1902) [n:127.0.0.1:34026_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
[junit4] 2> 803526 INFO (qtp2102923143-1902) [n:127.0.0.1:34026_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 15000ms;
[junit4] 2> 803526 INFO (qtp2102923143-1902) [n:127.0.0.1:34026_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 803559 INFO (qtp2102923143-1902) [n:127.0.0.1:34026_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=37, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=1.8720703125, noCFSRatio=0.0]
[junit4] 2> 803688 INFO (qtp2102923143-1902) [n:127.0.0.1:34026_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.s.SolrIndexSearcher Opening [Searcher@14b2c937[collection1_shard2_replica_n61] main]
[junit4] 2> 803726 INFO (qtp2102923143-1902) [n:127.0.0.1:34026_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 803726 INFO (qtp2102923143-1902) [n:127.0.0.1:34026_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 803727 INFO (qtp2102923143-1902) [n:127.0.0.1:34026_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 803744 INFO (searcherExecutor-694-thread-1-processing-n:127.0.0.1:34026_ x:collection1_shard2_replica_n61 s:shard2 c:collection1) [n:127.0.0.1:34026_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.SolrCore [collection1_shard2_replica_n61] Registered new searcher Searcher@14b2c937[collection1_shard2_replica_n61] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 803744 INFO (qtp2102923143-1902) [n:127.0.0.1:34026_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1583580510228054016
[junit4] 2> 803765 INFO (qtp2102923143-1902) [n:127.0.0.1:34026_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 803765 INFO (qtp2102923143-1902) [n:127.0.0.1:34026_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 803765 INFO (qtp2102923143-1902) [n:127.0.0.1:34026_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:34026/collection1_shard2_replica_n61/
[junit4] 2> 803765 INFO (qtp2102923143-1902) [n:127.0.0.1:34026_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 803765 INFO (qtp2102923143-1902) [n:127.0.0.1:34026_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.SyncStrategy http://127.0.0.1:34026/collection1_shard2_replica_n61/ has no replicas
[junit4] 2> 803765 INFO (qtp2102923143-1902) [n:127.0.0.1:34026_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 803816 INFO (qtp2102923143-1902) [n:127.0.0.1:34026_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:34026/collection1_shard2_replica_n61/ shard2
[junit4] 2> 803831 INFO (zkCallback-280-thread-1-processing-n:127.0.0.1:34026_) [n:127.0.0.1:34026_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 803872 INFO (qtp2102923143-1902) [n:127.0.0.1:34026_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 803873 INFO (qtp2102923143-1902) [n:127.0.0.1:34026_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n61&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=4729
[junit4] 2> 803909 INFO (qtp2102923143-1905) [n:127.0.0.1:34026_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:34026_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2} status=0 QTime=4875
[junit4] 2> 804014 INFO (zkCallback-280-thread-1-processing-n:127.0.0.1:34026_) [n:127.0.0.1:34026_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 805135 INFO (OverseerCollectionConfigSetProcessor-98973779555385348-127.0.0.1:38059_-n_0000000000) [n:127.0.0.1:38059_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 805977 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_B5660C1FA2E71586-001/shard-2-001 of type NRT
[junit4] 2> 805978 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 806136 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6a174969{/,null,AVAILABLE}
[junit4] 2> 806136 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@635a13a0{HTTP/1.1,[http/1.1]}{127.0.0.1:43660}
[junit4] 2> 806136 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.e.j.s.Server Started @818876ms
[junit4] 2> 806136 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://lucene2-us-west.apache.org:44071/hdfs__lucene2-us-west.apache.org_44071__home_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-master_checkout_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_B5660C1FA2E71586-001_tempDir-002_jetty2, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/, hostPort=43660, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_B5660C1FA2E71586-001/shard-2-001/cores}
[junit4] 2> 806137 ERROR (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 806156 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 806156 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 806156 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 806156 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-11-09T09:34:22.978Z
[junit4] 2> 806340 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 806340 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_B5660C1FA2E71586-001/shard-2-001/solr.xml
[junit4] 2> 806356 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 806356 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 806357 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@58af9533, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 806397 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33407/solr
[junit4] 2> 806582 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [n:127.0.0.1:43660_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 806616 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [n:127.0.0.1:43660_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 806618 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [n:127.0.0.1:43660_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43660_
[junit4] 2> 806620 INFO (zkCallback-280-thread-1-processing-n:127.0.0.1:34026_) [n:127.0.0.1:34026_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 806766 INFO (zkCallback-267-thread-2-processing-n:127.0.0.1:38059_) [n:127.0.0.1:38059_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 806774 INFO (zkCallback-275-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 806827 INFO (zkCallback-286-thread-1-processing-n:127.0.0.1:43660_) [n:127.0.0.1:43660_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 807311 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [n:127.0.0.1:43660_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58af9533
[junit4] 2> 807401 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [n:127.0.0.1:43660_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58af9533
[junit4] 2> 807402 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [n:127.0.0.1:43660_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58af9533
[junit4] 2> 807494 INFO (TEST-HdfsChaosMonkeySafeLeaderTest.test-seed#[B5660C1FA2E71586]) [n:127.0.0.1:43660_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_B5660C1FA2E71586-001/shard-2-001/cores
[junit4] 2> 807840 INFO (qtp2102923143-1906) [n:127.0.0.1:34026_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:43660_&action=ADDREPLICA&collection=collection1&shard=shard3&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 807909 INFO (OverseerThreadFactory-680-thread-4-processing-n:127.0.0.1:38059_) [n:127.0.0.1:38059_ ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:43660_ for creating new replica
[junit4] 2> 807971 INFO (qtp467418219-1943) [n:127.0.0.1:43660_ ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard3_replica_n63&action=CREATE&collection=collection1&shard=shard3&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 807971 INFO (qtp467418219-1943) [n:127.0.0.1:43660_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 807994 INFO (zkCallback-280-thread-1-processing-n:127.0.0.1:34026_) [n:127.0.0.1:34026_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 808082 INFO (qtp467418219-1943) [n:127.0.0.1:43660_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 808476 INFO (qtp467418219-1943) [n:127.0.0.1:43660_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.s.IndexSchema [collection1_shard3_replica_n63] Schema name=test
[junit4] 2> 809416 INFO (qtp467418219-1943) [n:127.0.0.1:43660_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 809621 INFO (qtp467418219-1943) [n:127.0.0.1:43660_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard3_replica_n63' using configuration from collection collection1, trusted=true
[junit4] 2> 809658 INFO (qtp467418219-1943) [n:127.0.0.1:43660_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard3.replica_n63' (registry 'solr.core.collection1.shard3.replica_n63') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@58af9533
[junit4] 2> 809707 INFO (qtp467418219-1943) [n:127.0.0.1:43660_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://lucene2-us-west.apache.org:44071/solr_hdfs_home
[junit4] 2> 809707 INFO (qtp467418219-1943) [n:127.0.0.1:43660_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 809707 INFO (qtp467418219-1943) [n:127.0.0.1:43660_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 809708 INFO (qtp467418219-1943) [n:127.0.0.1:43660_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.SolrCore [[collection1_shard3_replica_n63] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest_B5660C1FA2E71586-001/shard-2-001/cores/collection1_shard3_replica_n63], dataDir=[hdfs://lucene2-us-west.apache.org:44071/solr_hdfs_home/collection1/core_node64/data/]
[junit4] 2> 809709 INFO (qtp467418219-1943) [n:127.0.0.1:43660_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:44071/solr_hdfs_home/collection1/core_node64/data/snapshot_metadata
[junit4] 2> 809799 INFO (qtp467418219-1943) [n:127.0.0.1:43660_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 809799 INFO (qtp467418219-1943) [n:127.0.0.1:43660_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 809799 INFO (qtp467418219-1943) [n:127.0.0.1:43660_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 809959 INFO (qtp467418219-1943) [n:127.0.0.1:43660_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 809979 INFO (qtp467418219-1943) [n:127.0.0.1:43660_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:44071/solr_hdfs_home/collection1/core_node64/data
[junit4] 2> 810419 INFO (qtp467418219-1943) [n:127.0.0.1:43660_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:44071/solr_hdfs_home/collection1/core_node64/data/index
[junit4] 2> 810457 INFO (qtp467418219-1943) [n:127.0.0.1:43660_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
[junit4] 2> 810457 INFO (qtp467418219-1943) [n:127.0.0.1:43660_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
[junit4] 2> 810457 INFO (qtp467418219-1943) [n:127.0.0.1:43660_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
[junit4] 2> 810638 INFO (qtp467418219-1943) [n:127.0.0.1:43660_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.s.b.BlockDirectory Block cache on write is disabled
[junit4] 2> 810639 INFO (qtp467418219-1943) [n:127.0.0.1:43660_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@5eb6d1f5
[junit4] 2> 811369 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:34176 is added to blk_1073741827_1003{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-fe3bf6de-44cb-4c0a-860c-d3dc1b75fcf1:NORMAL:127.0.0.1:34876|RBW], ReplicaUC[[DISK]DS-5b1ef6fc-b102-4528-bfb9-dccb02c69102:NORMAL:127.0.0.1:34176|RBW]]} size 0
[junit4] 2> 811380 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:34876 is added to blk_1073741827_1003 size 69
[junit4] 2> 811694 WARN (qtp467418219-1943) [n:127.0.0.1:43660_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 812455 INFO (qtp467418219-1943)
[...truncated too long message...]
ster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/ocptest4/state.json] for collection [ocptest4] has occurred - updating... (live nodes size: [0])
[junit4] 2> 4305856 INFO (zkCallback-1130-thread-4-processing-n:127.0.0.1:40705_gk%2Fq) [n:127.0.0.1:40705_gk%2Fq ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/ocptest1/state.json] for collection [ocptest1] has occurred - updating... (live nodes size: [0])
[junit4] 2> 4305871 INFO (coreCloseExecutor-3299-thread-6) [n:127.0.0.1:40705_gk%2Fq c:ocptest_shardsplit s:shard2 r:core_node6 x:ocptest_shardsplit_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.ocptest_shardsplit.shard2.leader, tag=1373230316
[junit4] 2> 4305871 INFO (coreCloseExecutor-3299-thread-7) [n:127.0.0.1:40705_gk%2Fq c:ocptest_shardsplit2 s:shard3 r:core_node8 x:ocptest_shardsplit2_shard3_replica_n3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.ocptest_shardsplit2.shard3.replica_n3, tag=1072526499
[junit4] 2> 4305872 INFO (coreCloseExecutor-3299-thread-7) [n:127.0.0.1:40705_gk%2Fq c:ocptest_shardsplit2 s:shard3 r:core_node8 x:ocptest_shardsplit2_shard3_replica_n3] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2d8956b2: rootName = null, domain = solr.core.ocptest_shardsplit2.shard3.replica_n3, service url = null, agent id = null] for registry solr.core.ocptest_shardsplit2.shard3.replica_n3 / com.codahale.metrics.MetricRegistry@280e126a
[junit4] 2> 4305890 INFO (coreCloseExecutor-3299-thread-7) [n:127.0.0.1:40705_gk%2Fq c:ocptest_shardsplit2 s:shard3 r:core_node8 x:ocptest_shardsplit2_shard3_replica_n3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.ocptest_shardsplit2.shard3.leader, tag=1072526499
[junit4] 2> 4305890 INFO (coreCloseExecutor-3299-thread-1) [n:127.0.0.1:40705_gk%2Fq c:collection1 s:shard1 r:core_node48 x:collection1_shard1_replica_n47] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard1.leader, tag=1323348468
[junit4] 2> 4305890 INFO (coreCloseExecutor-3299-thread-1) [n:127.0.0.1:40705_gk%2Fq c:collection1 s:shard1 r:core_node48 x:collection1_shard1_replica_n47] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter close.
[junit4] 2> 4305890 INFO (coreCloseExecutor-3299-thread-1) [n:127.0.0.1:40705_gk%2Fq c:collection1 s:shard1 r:core_node48 x:collection1_shard1_replica_n47] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@5b78e9e1 commitCommandVersion:0
[junit4] 2> 4306095 INFO (TEST-MultiThreadedOCPTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.c.Overseer Overseer (id=98974004827062290-127.0.0.1:40705_gk%2Fq-n_0000000004) closing
[junit4] 2> 4306096 INFO (OverseerStateUpdate-98974004827062290-127.0.0.1:40705_gk%2Fq-n_0000000004) [n:127.0.0.1:40705_gk%2Fq ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:40705_gk%2Fq
[junit4] 2> 4306106 WARN (OverseerAutoScalingTriggerThread-98974004827062290-127.0.0.1:40705_gk%2Fq-n_0000000004) [n:127.0.0.1:40705_gk%2Fq ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 4306118 INFO (TEST-MultiThreadedOCPTest.test-seed#[B5660C1FA2E71586]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@24dc3b7b{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 4306118 INFO (TEST-MultiThreadedOCPTest.test-seed#[B5660C1FA2E71586]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@7735a1c8{/gk/q,null,UNAVAILABLE}
[junit4] 2> 4306118 ERROR (TEST-MultiThreadedOCPTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 4306119 INFO (TEST-MultiThreadedOCPTest.test-seed#[B5660C1FA2E71586]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:42739 42739
[junit4] 2> 4306199 INFO (Thread-4346) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:42739 42739
[junit4] 2> 4306199 WARN (Thread-4346) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 7 /solr/aliases.json
[junit4] 2> 5 /solr/security.json
[junit4] 2> 5 /solr/configs/conf1
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 7 /solr/clusterprops.json
[junit4] 2> 7 /solr/clusterstate.json
[junit4] 2> 4 /solr/collections/ocptest_shardsplit2/state.json
[junit4] 2> 4 /solr/collections/ocptest1/state.json
[junit4] 2> 4 /solr/collections/collection1/state.json
[junit4] 2> 4 /solr/collections/ocptest3/state.json
[junit4] 2> 4 /solr/collections/ocptest2/state.json
[junit4] 2> 4 /solr/autoscaling.json
[junit4] 2> 4 /solr/collections/ocptest4/state.json
[junit4] 2> 4 /solr/collections/ocptest_shardsplit/state.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 7 /solr/live_nodes
[junit4] 2> 7 /solr/collections
[junit4] 2> 4 /solr/overseer/queue
[junit4] 2> 4 /solr/autoscaling/events/.auto_add_replicas
[junit4] 2> 4 /solr/overseer/queue-work
[junit4] 2> 3 /solr/overseer/collection-queue-work
[junit4] 2>
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=MultiThreadedOCPTest -Dtests.method=test -Dtests.seed=B5660C1FA2E71586 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=ja -Dtests.timezone=Europe/Andorra -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] FAILURE 99.3s J2 | MultiThreadedOCPTest.test <<<
[junit4] > Throwable #1: java.lang.AssertionError
[junit4] > at __randomizedtesting.SeedInfo.seed([B5660C1FA2E71586:3D3233C50C1B787E]:0)
[junit4] > at org.apache.solr.cloud.MultiThreadedOCPTest.testFillWorkQueue(MultiThreadedOCPTest.java:112)
[junit4] > at org.apache.solr.cloud.MultiThreadedOCPTest.test(MultiThreadedOCPTest.java:67)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.MultiThreadedOCPTest_B5660C1FA2E71586-001
[junit4] 2> Nov 09, 2017 10:32:41 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {multiDefault=PostingsFormat(name=Asserting), id=FST50}, docValues:{range_facet_l_dv=DocValuesFormat(name=Memory), _version_=DocValuesFormat(name=Lucene70), multiDefault=DocValuesFormat(name=Lucene70), intDefault=DocValuesFormat(name=Lucene70), id_i1=DocValuesFormat(name=Lucene70), range_facet_i_dv=DocValuesFormat(name=Lucene70), id=DocValuesFormat(name=Memory), intDvoDefault=DocValuesFormat(name=Memory), timestamp=DocValuesFormat(name=Lucene70), range_facet_l=DocValuesFormat(name=Lucene70)}, maxPointsInLeafNode=1406, maxMBSortInHeap=6.945053174078652, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@2e24329d), locale=ja, timezone=Europe/Andorra
[junit4] 2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=1,free=143430704,total=525860864
[junit4] 2> NOTE: All tests run in this JVM: [TestSolrDeletionPolicy2, SuggesterWFSTTest, ClusterStateTest, TestLMJelinekMercerSimilarityFactory, TestDocTermOrds, SimpleMLTQParserTest, TestIndexSearcher, ZkStateReaderTest, CollectionsAPIAsyncDistributedZkTest, AliasIntegrationTest, EchoParamsTest, TestRandomFlRTGCloud, TestDeleteCollectionOnDownNodes, TestCollectionAPI, TestManagedSchema, TestMaxTokenLenTokenizer, PropertiesRequestHandlerTest, ManagedSchemaRoundRobinCloudTest, DistributedExpandComponentTest, TestRealTimeGet, AsyncCallRequestStatusResponseTest, ReplaceNodeTest, MetricsConfigTest, DefaultValueUpdateProcessorTest, CloneFieldUpdateProcessorFactoryTest, TestDefaultStatsCache, UniqFieldsUpdateProcessorFactoryTest, ReplicaListTransformerTest, DistributedSuggestComponentTest, DistributedFacetPivotLongTailTest, TestConfigOverlay, TestInPlaceUpdatesStandalone, TestOverriddenPrefixQueryForCustomFieldType, CdcrRequestHandlerTest, TestFieldTypeCollectionResource, TestLockTree, AnalyticsQueryTest, TestDistributedGrouping, TestConfigReload, TestFunctionQuery, ConfigSetsAPITest, TestReplicaProperties, HttpPartitionTest, HdfsDirectoryFactoryTest, TestUseDocValuesAsStored, TestPointFields, HdfsThreadLeakTest, TestSizeLimitedDistributedMap, TestExpandComponent, DeleteStatusTest, TestSubQueryTransformer, TestBulkSchemaAPI, CollectionReloadTest, TestSearcherReuse, TestShardHandlerFactory, HdfsCollectionsAPIDistributedZkTest, CoreSorterTest, SuggesterTest, TestFieldCacheSort, TestComplexPhraseQParserPlugin, TestGeoJSONResponseWriter, TestStandardQParsers, PeerSyncReplicationTest, MultiThreadedOCPTest]
[junit4] Completed [689/749 (3!)] on J2 in 99.81s, 1 test, 1 failure <<< FAILURES!
[...truncated 47388 lines...]
[JENKINS] Lucene-Solr-NightlyTests-master - Build # 1415 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-master/1415/
13 tests failed.
FAILED: org.apache.lucene.spatial3d.TestGeo3DPoint.testRandomBig
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([892A66AA6B2CC4E7]:0)
FAILED: junit.framework.TestSuite.org.apache.lucene.spatial3d.TestGeo3DPoint
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([892A66AA6B2CC4E7]:0)
FAILED: org.apache.lucene.codecs.asserting.TestAssertingStoredFieldsFormat.testRamBytesUsed
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([3C62B26DB1A08402]:0)
FAILED: junit.framework.TestSuite.org.apache.lucene.codecs.asserting.TestAssertingStoredFieldsFormat
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([3C62B26DB1A08402]:0)
FAILED: org.apache.lucene.codecs.asserting.TestAssertingTermVectorsFormat.testRamBytesUsed
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([3C62B26DB1A08402]:0)
FAILED: junit.framework.TestSuite.org.apache.lucene.codecs.asserting.TestAssertingTermVectorsFormat
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([3C62B26DB1A08402]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest
Error Message:
8 threads leaked from SUITE scope at org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest: 1) Thread[id=5092, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at java.lang.Thread.sleep(Native Method) at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135) 2) Thread[id=5093, name=StoppableCommitThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at java.lang.Thread.sleep(Native Method) at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:111) at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:70) at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:332) at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1184) at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1170) at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:692) at org.apache.solr.client.solrj.impl.CloudSolrClient.getDocCollection(CloudSolrClient.java:1198) at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:840) at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:808) at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194) at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:524) at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:544) at org.apache.solr.cloud.StoppableCommitThread.run(StoppableCommitThread.java:49) 3) Thread[id=5336, name=zkCallback-317-thread-2, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=5175, name=zkCallback-317-thread-1, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=5337, name=zkCallback-317-thread-3, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 6) Thread[id=5174, name=StoppableIndexingThread-EventThread, state=WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501) 7) Thread[id=5091, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at java.lang.Thread.sleep(Native Method) at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135) 8) Thread[id=5173, name=StoppableIndexingThread-SendThread(127.0.0.1:43035), state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 8 threads leaked from SUITE scope at org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest:
1) Thread[id=5092, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135)
2) Thread[id=5093, name=StoppableCommitThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:111)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:70)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:332)
at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1184)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1170)
at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:692)
at org.apache.solr.client.solrj.impl.CloudSolrClient.getDocCollection(CloudSolrClient.java:1198)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:840)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:808)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:524)
at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:544)
at org.apache.solr.cloud.StoppableCommitThread.run(StoppableCommitThread.java:49)
3) Thread[id=5336, name=zkCallback-317-thread-2, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
4) Thread[id=5175, name=zkCallback-317-thread-1, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
5) Thread[id=5337, name=zkCallback-317-thread-3, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
6) Thread[id=5174, name=StoppableIndexingThread-EventThread, state=WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:501)
7) Thread[id=5091, name=StoppableIndexingThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:135)
8) Thread[id=5173, name=StoppableIndexingThread-SendThread(127.0.0.1:43035), state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)
at __randomizedtesting.SeedInfo.seed([8F6AE201CD5CA398]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=5092, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at java.security.AccessController.doPrivileged(Native Method) at com.carrotsearch.ant.tasks.junit4.events.Serializer.flushQueue(Serializer.java:114) at com.carrotsearch.ant.tasks.junit4.events.Serializer.serialize(Serializer.java:99) at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain$3$2.write(SlaveMain.java:472) at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) at java.io.PrintStream.flush(PrintStream.java:338) at java.io.FilterOutputStream.flush(FilterOutputStream.java:140) at java.io.PrintStream.write(PrintStream.java:482) at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221) at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291) at sun.nio.cs.StreamEncoder.flushBuffer(StreamEncoder.java:104) at java.io.OutputStreamWriter.flushBuffer(OutputStreamWriter.java:185) at java.io.PrintStream.write(PrintStream.java:527) at java.io.PrintStream.print(PrintStream.java:669) at java.io.PrintStream.println(PrintStream.java:823) at java.lang.Throwable$WrappedPrintStream.println(Throwable.java:748) at java.lang.Throwable.printStackTrace(Throwable.java:658) at java.lang.Throwable.printStackTrace(Throwable.java:643) at java.lang.Throwable.printStackTrace(Throwable.java:634) at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:121) 2) Thread[id=5336, name=zkCallback-317-thread-2, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=5175, name=zkCallback-317-thread-1, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=5337, name=zkCallback-317-thread-3, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=5091, name=StoppableIndexingThread, state=BLOCKED, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at java.io.PrintStream.println(PrintStream.java:805) at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:120) 6) Thread[id=5173, name=StoppableIndexingThread-SendThread(127.0.0.1:43035), state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=5092, name=StoppableIndexingThread, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at java.security.AccessController.doPrivileged(Native Method)
at com.carrotsearch.ant.tasks.junit4.events.Serializer.flushQueue(Serializer.java:114)
at com.carrotsearch.ant.tasks.junit4.events.Serializer.serialize(Serializer.java:99)
at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain$3$2.write(SlaveMain.java:472)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
at java.io.PrintStream.flush(PrintStream.java:338)
at java.io.FilterOutputStream.flush(FilterOutputStream.java:140)
at java.io.PrintStream.write(PrintStream.java:482)
at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
at sun.nio.cs.StreamEncoder.flushBuffer(StreamEncoder.java:104)
at java.io.OutputStreamWriter.flushBuffer(OutputStreamWriter.java:185)
at java.io.PrintStream.write(PrintStream.java:527)
at java.io.PrintStream.print(PrintStream.java:669)
at java.io.PrintStream.println(PrintStream.java:823)
at java.lang.Throwable$WrappedPrintStream.println(Throwable.java:748)
at java.lang.Throwable.printStackTrace(Throwable.java:658)
at java.lang.Throwable.printStackTrace(Throwable.java:643)
at java.lang.Throwable.printStackTrace(Throwable.java:634)
at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:121)
2) Thread[id=5336, name=zkCallback-317-thread-2, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=5175, name=zkCallback-317-thread-1, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
4) Thread[id=5337, name=zkCallback-317-thread-3, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
5) Thread[id=5091, name=StoppableIndexingThread, state=BLOCKED, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at java.io.PrintStream.println(PrintStream.java:805)
at org.apache.solr.cloud.StoppableIndexingThread.run(StoppableIndexingThread.java:120)
6) Thread[id=5173, name=StoppableIndexingThread-SendThread(127.0.0.1:43035), state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderWithPullReplicasTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:997)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1060)
at __randomizedtesting.SeedInfo.seed([8F6AE201CD5CA398]:0)
FAILED: org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest.test
Error Message:
The Monkey ran for over 45 seconds and no jetties were stopped - this is worth investigating!
Stack Trace:
java.lang.AssertionError: The Monkey ran for over 45 seconds and no jetties were stopped - this is worth investigating!
at __randomizedtesting.SeedInfo.seed([8F6AE201CD5CA398:73EDDDB63A0CE60]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.ChaosMonkey.stopTheMonkey(ChaosMonkey.java:587)
at org.apache.solr.cloud.ChaosMonkeySafeLeaderWithPullReplicasTest.test(ChaosMonkeySafeLeaderWithPullReplicasTest.java:174)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.SystemLogListenerTest.test
Error Message:
Trigger was not fired
Stack Trace:
java.lang.AssertionError: Trigger was not fired
at __randomizedtesting.SeedInfo.seed([8F6AE201CD5CA398:73EDDDB63A0CE60]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.SystemLogListenerTest.test(SystemLogListenerTest.java:151)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.hdfs.HdfsBasicDistributedZk2Test.test
Error Message:
.response.numFound:36!=67
Stack Trace:
junit.framework.AssertionFailedError: .response.numFound:36!=67
at __randomizedtesting.SeedInfo.seed([8F6AE201CD5CA398:73EDDDB63A0CE60]:0)
at junit.framework.Assert.fail(Assert.java:50)
at org.apache.solr.BaseDistributedSearchTestCase.compareSolrResponses(BaseDistributedSearchTestCase.java:913)
at org.apache.solr.BaseDistributedSearchTestCase.compareResponses(BaseDistributedSearchTestCase.java:940)
at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:607)
at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:587)
at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:566)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkQueries(AbstractFullDistribZkTestBase.java:921)
at org.apache.solr.cloud.BasicDistributedZk2Test.test(BasicDistributedZk2Test.java:102)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload
Error Message:
expected:<[{indexVersion=1510089220724,generation=2,filelist=[_68.cfe, _68.cfs, _68.si, _69.fdt, _69.fdx, _69.fnm, _69.nvd, _69.nvm, _69.si, _69_Direct_0.doc, _69_Direct_0.tim, _69_Direct_0.tip, _6a.cfe, _6a.cfs, _6a.si, _6b.cfe, _6b.cfs, _6b.si, _6c.cfe, _6c.cfs, _6c.si, _6d.cfe, _6d.cfs, _6d.si, _6e.cfe, _6e.cfs, _6e.si, _6f.cfe, _6f.cfs, _6f.si, _6g.cfe, _6g.cfs, _6g.si, _6h.cfe, _6h.cfs, _6h.si, _6i.cfe, _6i.cfs, _6i.si, _6j.cfe, _6j.cfs, _6j.si, _6k.cfe, _6k.cfs, _6k.si, _6l.cfe, _6l.cfs, _6l.si, _6m.cfe, _6m.cfs, _6m.si, _6o.cfe, _6o.cfs, _6o.si, _6p.cfe, _6p.cfs, _6p.si, segments_2]}]> but was:<[{indexVersion=1510089220724,generation=2,filelist=[_68.cfe, _68.cfs, _68.si, _69.fdt, _69.fdx, _69.fnm, _69.nvd, _69.nvm, _69.si, _69_Direct_0.doc, _69_Direct_0.tim, _69_Direct_0.tip, _6a.cfe, _6a.cfs, _6a.si, _6b.cfe, _6b.cfs, _6b.si, _6c.cfe, _6c.cfs, _6c.si, _6d.cfe, _6d.cfs, _6d.si, _6e.cfe, _6e.cfs, _6e.si, _6f.cfe, _6f.cfs, _6f.si, _6g.cfe, _6g.cfs, _6g.si, _6h.cfe, _6h.cfs, _6h.si, _6i.cfe, _6i.cfs, _6i.si, _6j.cfe, _6j.cfs, _6j.si, _6k.cfe, _6k.cfs, _6k.si, _6l.cfe, _6l.cfs, _6l.si, _6m.cfe, _6m.cfs, _6m.si, _6o.cfe, _6o.cfs, _6o.si, _6p.cfe, _6p.cfs, _6p.si, segments_2]}, {indexVersion=1510089220724,generation=3,filelist=[_6m.cfe, _6m.cfs, _6m.si, _6n.fdt, _6n.fdx, _6n.fnm, _6n.nvd, _6n.nvm, _6n.si, _6n_Direct_0.doc, _6n_Direct_0.tim, _6n_Direct_0.tip, _6o.cfe, _6o.cfs, _6o.si, _6p.cfe, _6p.cfs, _6p.si, segments_3]}]>
Stack Trace:
java.lang.AssertionError: expected:<[{indexVersion=1510089220724,generation=2,filelist=[_68.cfe, _68.cfs, _68.si, _69.fdt, _69.fdx, _69.fnm, _69.nvd, _69.nvm, _69.si, _69_Direct_0.doc, _69_Direct_0.tim, _69_Direct_0.tip, _6a.cfe, _6a.cfs, _6a.si, _6b.cfe, _6b.cfs, _6b.si, _6c.cfe, _6c.cfs, _6c.si, _6d.cfe, _6d.cfs, _6d.si, _6e.cfe, _6e.cfs, _6e.si, _6f.cfe, _6f.cfs, _6f.si, _6g.cfe, _6g.cfs, _6g.si, _6h.cfe, _6h.cfs, _6h.si, _6i.cfe, _6i.cfs, _6i.si, _6j.cfe, _6j.cfs, _6j.si, _6k.cfe, _6k.cfs, _6k.si, _6l.cfe, _6l.cfs, _6l.si, _6m.cfe, _6m.cfs, _6m.si, _6o.cfe, _6o.cfs, _6o.si, _6p.cfe, _6p.cfs, _6p.si, segments_2]}]> but was:<[{indexVersion=1510089220724,generation=2,filelist=[_68.cfe, _68.cfs, _68.si, _69.fdt, _69.fdx, _69.fnm, _69.nvd, _69.nvm, _69.si, _69_Direct_0.doc, _69_Direct_0.tim, _69_Direct_0.tip, _6a.cfe, _6a.cfs, _6a.si, _6b.cfe, _6b.cfs, _6b.si, _6c.cfe, _6c.cfs, _6c.si, _6d.cfe, _6d.cfs, _6d.si, _6e.cfe, _6e.cfs, _6e.si, _6f.cfe, _6f.cfs, _6f.si, _6g.cfe, _6g.cfs, _6g.si, _6h.cfe, _6h.cfs, _6h.si, _6i.cfe, _6i.cfs, _6i.si, _6j.cfe, _6j.cfs, _6j.si, _6k.cfe, _6k.cfs, _6k.si, _6l.cfe, _6l.cfs, _6l.si, _6m.cfe, _6m.cfs, _6m.si, _6o.cfe, _6o.cfs, _6o.si, _6p.cfe, _6p.cfs, _6p.si, segments_2]}, {indexVersion=1510089220724,generation=3,filelist=[_6m.cfe, _6m.cfs, _6m.si, _6n.fdt, _6n.fdx, _6n.fnm, _6n.nvd, _6n.nvm, _6n.si, _6n_Direct_0.doc, _6n_Direct_0.tim, _6n_Direct_0.tip, _6o.cfe, _6o.cfs, _6o.si, _6p.cfe, _6p.cfs, _6p.si, segments_3]}]>
at __randomizedtesting.SeedInfo.seed([8F6AE201CD5CA398:AABDF931BD14AD9B]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:147)
at org.apache.solr.handler.TestReplicationHandler.doTestReplicateAfterCoreReload(TestReplicationHandler.java:1277)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.update.CdcrUpdateLogTest.testSubReader
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([8F6AE201CD5CA398:C2B26862BD8B724A]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.update.CdcrUpdateLogTest.testSubReader(CdcrUpdateLogTest.java:583)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 3255 lines...]
[junit4] Suite: org.apache.lucene.codecs.asserting.TestAssertingTermVectorsFormat
[junit4] 2> 11 07, 2017 10:22:00 午後 com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
[junit4] 2> WARNING: Suite execution timed out: org.apache.lucene.codecs.asserting.TestAssertingTermVectorsFormat
[junit4] 2> 1) Thread[id=1, name=main, state=WAITING, group=main]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Thread.join(Thread.java:1252)
[junit4] 2> at java.lang.Thread.join(Thread.java:1326)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:636)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:493)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:251)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:368)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:13)
[junit4] 2> 2) Thread[id=14, name=SUITE-TestAssertingTermVectorsFormat-seed#[3C62B26DB1A08402], state=RUNNABLE, group=TGRP-TestAssertingTermVectorsFormat]
[junit4] 2> at java.lang.Thread.getStackTrace(Thread.java:1559)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:696)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:693)
[junit4] 2> at java.security.AccessController.doPrivileged(Native Method)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getStackTrace(ThreadLeakControl.java:693)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getThreadsWithTraces(ThreadLeakControl.java:709)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:689)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:65)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:415)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:705)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:139)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:626)
[junit4] 2> 3) Thread[id=9, name=JUnit4-serializer-daemon, state=TIMED_WAITING, group=main]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:50)
[junit4] 2> 4) Thread[id=15, name=TEST-TestAssertingTermVectorsFormat.testRamBytesUsed-seed#[3C62B26DB1A08402], state=RUNNABLE, group=TGRP-TestAssertingTermVectorsFormat]
[junit4] 2> at org.apache.lucene.codecs.compressing.CompressingTermVectorsReader.get(CompressingTermVectorsReader.java:373)
[junit4] 2> at org.apache.lucene.codecs.asserting.AssertingTermVectorsFormat$AssertingTermVectorsReader.get(AssertingTermVectorsFormat.java:71)
[junit4] 2> at org.apache.lucene.codecs.TermVectorsWriter.merge(TermVectorsWriter.java:223)
[junit4] 2> at org.apache.lucene.index.SegmentMerger.mergeVectors(SegmentMerger.java:224)
[junit4] 2> at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:158)
[junit4] 2> at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4390)
[junit4] 2> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4030)
[junit4] 2> at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
[junit4] 2> at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2233)
[junit4] 2> at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5045)
[junit4] 2> at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1718)
[junit4] 2> at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1460)
[junit4] 2> at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRamBytesUsed(BaseIndexFileFormatTestCase.java:273)
[junit4] 2> at org.apache.lucene.index.BaseTermVectorsFormatTestCase.testRamBytesUsed(BaseTermVectorsFormatTestCase.java:71)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestAssertingTermVectorsFormat -Dtests.method=testRamBytesUsed -Dtests.seed=3C62B26DB1A08402 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=ja -Dtests.timezone=Indian/Christmas -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 7169s J1 | TestAssertingTermVectorsFormat.testRamBytesUsed <<<
[junit4] > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
[junit4] > at __randomizedtesting.SeedInfo.seed([3C62B26DB1A08402]:0)
[junit4] 2> 11 07, 2017 10:22:01 午後 com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> 11 07, 2017 10:22:21 午後 com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.lucene.codecs.asserting.TestAssertingTermVectorsFormat:
[junit4] 2> 1) Thread[id=15, name=TEST-TestAssertingTermVectorsFormat.testRamBytesUsed-seed#[3C62B26DB1A08402], state=RUNNABLE, group=TGRP-TestAssertingTermVectorsFormat]
[junit4] 2> at org.apache.lucene.store.DataInput.readVInt(DataInput.java:125)
[junit4] 2> at org.apache.lucene.codecs.lucene50.Lucene50PostingsReader$BlockPostingsEnum.refillPositions(Lucene50PostingsReader.java:618)
[junit4] 2> at org.apache.lucene.codecs.lucene50.Lucene50PostingsReader$BlockPostingsEnum.nextPosition(Lucene50PostingsReader.java:773)
[junit4] 2> at org.apache.lucene.index.FilterLeafReader$FilterPostingsEnum.nextPosition(FilterLeafReader.java:262)
[junit4] 2> at org.apache.lucene.index.AssertingLeafReader$AssertingPostingsEnum.nextPosition(AssertingLeafReader.java:403)
[junit4] 2> at org.apache.lucene.index.MappingMultiPostingsEnum.nextPosition(MappingMultiPostingsEnum.java:113)
[junit4] 2> at org.apache.lucene.codecs.asserting.AssertingPostingsFormat$AssertingFieldsConsumer.write(AssertingPostingsFormat.java:220)
[junit4] 2> at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:105)
[junit4] 2> at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsWriter.merge(PerFieldPostingsFormat.java:164)
[junit4] 2> at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:230)
[junit4] 2> at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:115)
[junit4] 2> at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4390)
[junit4] 2> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4030)
[junit4] 2> at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
[junit4] 2> at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2233)
[junit4] 2> at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5045)
[junit4] 2> at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1718)
[junit4] 2> at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1460)
[junit4] 2> at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRamBytesUsed(BaseIndexFileFormatTestCase.java:273)
[junit4] 2> at org.apache.lucene.index.BaseTermVectorsFormatTestCase.testRamBytesUsed(BaseTermVectorsFormatTestCase.java:71)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 11 07, 2017 10:22:21 午後 com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=15, name=TEST-TestAssertingTermVectorsFormat.testRamBytesUsed-seed#[3C62B26DB1A08402], state=RUNNABLE, group=TGRP-TestAssertingTermVectorsFormat]
[junit4] 2> 11 07, 2017 10:22:24 午後 com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> SEVERE: There are still zombie threads that couldn't be terminated:
[junit4] 2> 1) Thread[id=15, name=TEST-TestAssertingTermVectorsFormat.testRamBytesUsed-seed#[3C62B26DB1A08402], state=RUNNABLE, group=TGRP-TestAssertingTermVectorsFormat]
[junit4] 2> at org.apache.lucene.codecs.compressing.CompressingTermVectorsWriter$DocData.<init>(CompressingTermVectorsWriter.java:95)
[junit4] 2> at org.apache.lucene.codecs.compressing.CompressingTermVectorsWriter.addDocData(CompressingTermVectorsWriter.java:132)
[junit4] 2> at org.apache.lucene.codecs.compressing.CompressingTermVectorsWriter.startDocument(CompressingTermVectorsWriter.java:266)
[junit4] 2> at org.apache.lucene.codecs.asserting.AssertingTermVectorsFormat$AssertingTermVectorsWriter.startDocument(AssertingTermVectorsFormat.java:132)
[junit4] 2> at org.apache.lucene.codecs.TermVectorsWriter.addAllDocVectors(TermVectorsWriter.java:250)
[junit4] 2> at org.apache.lucene.codecs.TermVectorsWriter.merge(TermVectorsWriter.java:225)
[junit4] 2> at org.apache.lucene.index.SegmentMerger.mergeVectors(SegmentMerger.java:224)
[junit4] 2> at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:158)
[junit4] 2> at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4390)
[junit4] 2> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4030)
[junit4] 2> at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
[junit4] 2> at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2233)
[junit4] 2> at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5045)
[junit4] 2> at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1718)
[junit4] 2> at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1460)
[junit4] 2> at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRamBytesUsed(BaseIndexFileFormatTestCase.java:273)
[junit4] 2> at org.apache.lucene.index.BaseTermVectorsFormatTestCase.testRamBytesUsed(BaseTermVectorsFormatTestCase.java:71)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=1244, maxMBSortInHeap=5.17119862042723, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@67614ae9), locale=ja, timezone=Indian/Christmas
[junit4] 2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=2,free=134478872,total=300941312
[junit4] 2> NOTE: All tests run in this JVM: [TestGroupFiltering, TestVerboseFS, TestAssertingTermVectorsFormat]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestAssertingTermVectorsFormat -Dtests.seed=3C62B26DB1A08402 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=ja -Dtests.timezone=Indian/Christmas -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J1 | TestAssertingTermVectorsFormat (suite) <<<
[junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4] > at __randomizedtesting.SeedInfo.seed([3C62B26DB1A08402]:0)
[junit4] Completed [37/41 (1!)] on J1 in 7223.90s, 7 tests, 2 errors <<< FAILURES!
[...truncated 41 lines...]
[junit4] Suite: org.apache.lucene.codecs.asserting.TestAssertingStoredFieldsFormat
[junit4] 2> nov. 07, 2017 5:56:34 DU com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
[junit4] 2> WARNING: Suite execution timed out: org.apache.lucene.codecs.asserting.TestAssertingStoredFieldsFormat
[junit4] 2> 1) Thread[id=40, name=SUITE-TestAssertingStoredFieldsFormat-seed#[3C62B26DB1A08402], state=RUNNABLE, group=TGRP-TestAssertingStoredFieldsFormat]
[junit4] 2> at java.lang.Thread.getStackTrace(Thread.java:1559)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:696)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:693)
[junit4] 2> at java.security.AccessController.doPrivileged(Native Method)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getStackTrace(ThreadLeakControl.java:693)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getThreadsWithTraces(ThreadLeakControl.java:709)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:689)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:65)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:415)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:705)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:139)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:626)
[junit4] 2> 2) Thread[id=41, name=TEST-TestAssertingStoredFieldsFormat.testRamBytesUsed-seed#[3C62B26DB1A08402], state=RUNNABLE, group=TGRP-TestAssertingStoredFieldsFormat]
[junit4] 2> at org.apache.lucene.codecs.compressing.CompressingTermVectorsReader.get(CompressingTermVectorsReader.java:539)
[junit4] 2> at org.apache.lucene.codecs.asserting.AssertingTermVectorsFormat$AssertingTermVectorsReader.get(AssertingTermVectorsFormat.java:71)
[junit4] 2> at org.apache.lucene.codecs.TermVectorsWriter.merge(TermVectorsWriter.java:223)
[junit4] 2> at org.apache.lucene.index.SegmentMerger.mergeVectors(SegmentMerger.java:224)
[junit4] 2> at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:158)
[junit4] 2> at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4390)
[junit4] 2> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4030)
[junit4] 2> at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
[junit4] 2> at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2233)
[junit4] 2> at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5045)
[junit4] 2> at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1718)
[junit4] 2> at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1460)
[junit4] 2> at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRamBytesUsed(BaseIndexFileFormatTestCase.java:273)
[junit4] 2> at org.apache.lucene.index.BaseTermVectorsFormatTestCase.testRamBytesUsed(BaseTermVectorsFormatTestCase.java:71)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 3) Thread[id=1, name=main, state=WAITING, group=main]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Thread.join(Thread.java:1252)
[junit4] 2> at java.lang.Thread.join(Thread.java:1326)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:636)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:493)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:251)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:368)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:13)
[junit4] 2> 4) Thread[id=9, name=JUnit4-serializer-daemon, state=TIMED_WAITING, group=main]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:50)
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestAssertingStoredFieldsFormat -Dtests.method=testRamBytesUsed -Dtests.seed=3C62B26DB1A08402 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/test-data/enwiki.random.lines.txt -Dtests.locale=hu -Dtests.timezone=Europe/Riga -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 7188s J0 | TestAssertingStoredFieldsFormat.testRamBytesUsed <<<
[junit4] > Throwable #1: java.lang.Exception: Test abandoned because suite timeout was reached.
[junit4] > at __randomizedtesting.SeedInfo.seed([3C62B26DB1A08402]:0)
[junit4] 2> nov. 07, 2017 5:56:35 DU com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> nov. 07, 2017 5:56:55 DU com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.lucene.codecs.asserting.TestAssertingStoredFieldsFormat:
[junit4] 2> 1) Thread[id=41, name=TEST-TestAssertingStoredFieldsFormat.testRamBytesUsed-seed#[3C62B26DB1A08402], state=RUNNABLE, group=TGRP-TestAssertingStoredFieldsFormat]
[junit4] 2> at org.apache.lucene.store.DataInput.readVInt(DataInput.java:125)
[junit4] 2> at org.apache.lucene.codecs.compressing.CompressingTermVectorsReader.get(CompressingTermVectorsReader.java:332)
[junit4] 2> at org.apache.lucene.codecs.asserting.AssertingTermVectorsFormat$AssertingTermVectorsReader.get(AssertingTermVectorsFormat.java:71)
[junit4] 2> at org.apache.lucene.codecs.TermVectorsWriter.merge(TermVectorsWriter.java:223)
[junit4] 2> at org.apache.lucene.index.SegmentMerger.mergeVectors(SegmentMerger.java:224)
[junit4] 2> at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:158)
[junit4] 2> at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4390)
[junit4] 2> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4030)
[junit4] 2> at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:40)
[junit4] 2> at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:2233)
[junit4] 2> at org.apache.lucene.index.IndexWriter.processEvents(IndexWriter.java:5045)
[junit4] 2> at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1718)
[junit4] 2> at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1460)
[junit4] 2> at org.apache.lucene.index.BaseIndexFileFormatTestCase.testRamBytesUsed(BaseIndexFileFormatTestCase.java:273)
[junit4] 2> at org.apache.lucene.index.BaseTermVectorsFormatTestCase.testRamBytesUsed(BaseTermVectorsFormatTestCase.java:71)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2>
[...truncated too long message...]
ogUpdateProcessorFactory [collection1] webapp=null path=null params={versions=true&wt=json&indent=true}{add=[210 (1583444742595674116)]} 0 0
[junit4] 2> 2279069 INFO (TEST-CdcrUpdateLogTest.testRemoveOldLogsMultiplePointers-seed#[8F6AE201CD5CA398]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={versions=true&wt=json&indent=true}{add=[211 (1583444742595674117)]} 0 0
[junit4] 2> 2279069 INFO (TEST-CdcrUpdateLogTest.testRemoveOldLogsMultiplePointers-seed#[8F6AE201CD5CA398]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={versions=true&wt=json&indent=true}{add=[212 (1583444742595674118)]} 0 0
[junit4] 2> 2279069 INFO (TEST-CdcrUpdateLogTest.testRemoveOldLogsMultiplePointers-seed#[8F6AE201CD5CA398]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={versions=true&wt=json&indent=true}{add=[213 (1583444742595674119)]} 0 0
[junit4] 2> 2279069 INFO (TEST-CdcrUpdateLogTest.testRemoveOldLogsMultiplePointers-seed#[8F6AE201CD5CA398]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={versions=true&wt=json&indent=true}{add=[214 (1583444742595674120)]} 0 0
[junit4] 2> 2279069 INFO (TEST-CdcrUpdateLogTest.testRemoveOldLogsMultiplePointers-seed#[8F6AE201CD5CA398]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={versions=true&wt=json&indent=true}{add=[215 (1583444742595674121)]} 0 0
[junit4] 2> 2279070 INFO (TEST-CdcrUpdateLogTest.testRemoveOldLogsMultiplePointers-seed#[8F6AE201CD5CA398]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={versions=true&wt=json&indent=true}{add=[216 (1583444742595674122)]} 0 0
[junit4] 2> 2279070 INFO (TEST-CdcrUpdateLogTest.testRemoveOldLogsMultiplePointers-seed#[8F6AE201CD5CA398]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={versions=true&wt=json&indent=true}{add=[217 (1583444742596722688)]} 0 0
[junit4] 2> 2279070 INFO (TEST-CdcrUpdateLogTest.testRemoveOldLogsMultiplePointers-seed#[8F6AE201CD5CA398]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={versions=true&wt=json&indent=true}{add=[218 (1583444742596722689)]} 0 0
[junit4] 2> 2279070 INFO (TEST-CdcrUpdateLogTest.testRemoveOldLogsMultiplePointers-seed#[8F6AE201CD5CA398]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={versions=true&wt=json&indent=true}{add=[219 (1583444742596722690)]} 0 0
[junit4] 2> 2279070 INFO (TEST-CdcrUpdateLogTest.testRemoveOldLogsMultiplePointers-seed#[8F6AE201CD5CA398]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={versions=true&wt=json&indent=true}{add=[220 (1583444742596722691)]} 0 0
[junit4] 2> 2279070 INFO (TEST-CdcrUpdateLogTest.testRemoveOldLogsMultiplePointers-seed#[8F6AE201CD5CA398]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={versions=true&wt=json&indent=true}{add=[221 (1583444742596722692)]} 0 0
[junit4] 2> 2279070 INFO (TEST-CdcrUpdateLogTest.testRemoveOldLogsMultiplePointers-seed#[8F6AE201CD5CA398]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={versions=true&wt=json&indent=true}{add=[222 (1583444742596722693)]} 0 0
[junit4] 2> 2279070 INFO (TEST-CdcrUpdateLogTest.testRemoveOldLogsMultiplePointers-seed#[8F6AE201CD5CA398]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={versions=true&wt=json&indent=true}{add=[223 (1583444742596722694)]} 0 0
[junit4] 2> 2279070 INFO (TEST-CdcrUpdateLogTest.testRemoveOldLogsMultiplePointers-seed#[8F6AE201CD5CA398]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={versions=true&wt=json&indent=true}{add=[224 (1583444742596722695)]} 0 0
[junit4] 2> 2279070 INFO (TEST-CdcrUpdateLogTest.testRemoveOldLogsMultiplePointers-seed#[8F6AE201CD5CA398]) [ ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1583444742596722696,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2279070 INFO (TEST-CdcrUpdateLogTest.testRemoveOldLogsMultiplePointers-seed#[8F6AE201CD5CA398]) [ ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@50b4d1a4 commitCommandVersion:1583444742596722696
[junit4] 2> 2279374 INFO (TEST-CdcrUpdateLogTest.testRemoveOldLogsMultiplePointers-seed#[8F6AE201CD5CA398]) [ ] o.a.s.s.SolrIndexSearcher Opening [Searcher@7bb457cc[collection1] main]
[junit4] 2> 2279375 INFO (TEST-CdcrUpdateLogTest.testRemoveOldLogsMultiplePointers-seed#[8F6AE201CD5CA398]) [ ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2279375 INFO (searcherExecutor-2691-thread-1) [ ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@7bb457cc[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_w(8.0.0):c10) Uninverting(_x(8.0.0):c10) Uninverting(_y(8.0.0):c105) Uninverting(_z(8.0.0):c10) Uninverting(_10(8.0.0):c10) Uninverting(_11(8.0.0):c80)))}
[junit4] 2> 2279376 INFO (TEST-CdcrUpdateLogTest.testRemoveOldLogsMultiplePointers-seed#[8F6AE201CD5CA398]) [ ] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=null path=null params={}{commit=} 0 305
[junit4] 2> 2279376 INFO (TEST-CdcrUpdateLogTest.testRemoveOldLogsMultiplePointers-seed#[8F6AE201CD5CA398]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testRemoveOldLogsMultiplePointers
[junit4] 2> 2279376 INFO (SUITE-CdcrUpdateLogTest-seed#[8F6AE201CD5CA398]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> 2279376 INFO (SUITE-CdcrUpdateLogTest-seed#[8F6AE201CD5CA398]-worker) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1031524964
[junit4] 2> 2279376 INFO (SUITE-CdcrUpdateLogTest-seed#[8F6AE201CD5CA398]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 2279377 INFO (SUITE-CdcrUpdateLogTest-seed#[8F6AE201CD5CA398]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@29296c73: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@431de66f
[junit4] 2> 2279412 INFO (SUITE-CdcrUpdateLogTest-seed#[8F6AE201CD5CA398]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 2279412 INFO (SUITE-CdcrUpdateLogTest-seed#[8F6AE201CD5CA398]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5e8ffff: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@2073948c
[junit4] 2> 2279427 INFO (SUITE-CdcrUpdateLogTest-seed#[8F6AE201CD5CA398]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 2279427 INFO (SUITE-CdcrUpdateLogTest-seed#[8F6AE201CD5CA398]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@46b58a5d: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@266bb828
[junit4] 2> 2279434 INFO (coreCloseExecutor-2696-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@3310ebd2
[junit4] 2> 2279435 INFO (coreCloseExecutor-2696-thread-1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=856746962
[junit4] 2> 2279435 INFO (coreCloseExecutor-2696-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@332c02d4: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@630f0df4
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-master/checkout/solr/build/solr-core/test/J0/temp/solr.update.CdcrUpdateLogTest_8F6AE201CD5CA398-001
[junit4] 2> Nov 07, 2017 9:36:22 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Lucene70, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@449fea70), locale=sv, timezone=America/Tijuana
[junit4] 2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=1,free=269835272,total=531628032
[junit4] 2> NOTE: All tests run in this JVM: [AssignBackwardCompatibilityTest, TestRandomCollapseQParserPlugin, DateFieldTest, SystemLogListenerTest, SpatialFilterTest, QueryResultKeyTest, TestReplicationHandler, TestCollectionsAPIViaSolrCloudCluster, TestMiniSolrCloudClusterSSL, HdfsDirectoryTest, TestLegacyNumericUtils, CollectionStateFormat2Test, ForceLeaderTest, SolrJmxReporterCloudTest, LeaderInitiatedRecoveryOnCommitTest, PreAnalyzedUpdateProcessorTest, TestSort, TestSimpleQParserPlugin, ExternalFileFieldSortTest, TestFastOutputStream, CachingDirectoryFactoryTest, TestEmbeddedSolrServerSchemaAPI, DistribCursorPagingTest, ConnectionManagerTest, TermsComponentTest, TestLuceneIndexBackCompat, TestFieldCacheSortRandom, SpellCheckCollatorTest, SolrCloudExampleTest, TestCSVLoader, ResourceLoaderTest, TestImpersonationWithHadoopAuth, CacheHeaderTest, CdcrUpdateLogTest]
[junit4] Completed [576/749 (5!)] on J0 in 56.22s, 12 tests, 1 failure <<< FAILURES!
[...truncated 47810 lines...]