You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2019/04/04 14:29:50 UTC
[JENKINS] Lucene-Solr-NightlyTests-8.x - Build # 62 - Still
Unstable
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-8.x/62/
7 tests failed.
FAILED: org.apache.solr.update.TestInPlaceUpdatesDistrib.test
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([E3F3881478E51589]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.update.TestInPlaceUpdatesDistrib
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([E3F3881478E51589]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.update.TestInPlaceUpdatesDistrib
Error Message:
Captured an uncaught exception in thread: Thread[id=119832, name=replayUpdatesExecutor-20261-thread-1, state=RUNNABLE, group=TGRP-TestInPlaceUpdatesDistrib]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=119832, name=replayUpdatesExecutor-20261-thread-1, state=RUNNABLE, group=TGRP-TestInPlaceUpdatesDistrib]
Caused by: org.apache.solr.update.SolrCoreState$CoreIsClosedException
at __randomizedtesting.SeedInfo.seed([E3F3881478E51589]:0)
at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:2026)
at org.apache.solr.core.SolrCore.getRealtimeSearcher(SolrCore.java:2001)
at org.apache.solr.update.VersionInfo.getVersionFromIndex(VersionInfo.java:214)
at org.apache.solr.update.UpdateLog.lookupVersion(UpdateLog.java:1058)
at org.apache.solr.update.VersionInfo.lookupVersion(VersionInfo.java:203)
at org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:453)
at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:220)
at org.apache.solr.update.processor.DistributedZkUpdateProcessor.processAdd(DistributedZkUpdateProcessor.java:231)
at org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:103)
at org.apache.solr.update.UpdateLog$LogReplayer.lambda$execute$1(UpdateLog.java:2025)
at org.apache.solr.util.OrderedExecutor.lambda$execute$0(OrderedExecutor.java:69)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.update.TestInPlaceUpdatesDistrib
Error Message:
Captured an uncaught exception in thread: Thread[id=119833, name=replayUpdatesExecutor-20261-thread-2, state=RUNNABLE, group=TGRP-TestInPlaceUpdatesDistrib]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=119833, name=replayUpdatesExecutor-20261-thread-2, state=RUNNABLE, group=TGRP-TestInPlaceUpdatesDistrib]
Caused by: org.apache.solr.update.SolrCoreState$CoreIsClosedException
at __randomizedtesting.SeedInfo.seed([E3F3881478E51589]:0)
at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:2026)
at org.apache.solr.core.SolrCore.getRealtimeSearcher(SolrCore.java:2001)
at org.apache.solr.update.VersionInfo.getVersionFromIndex(VersionInfo.java:214)
at org.apache.solr.update.UpdateLog.lookupVersion(UpdateLog.java:1058)
at org.apache.solr.update.VersionInfo.lookupVersion(VersionInfo.java:203)
at org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:453)
at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:220)
at org.apache.solr.update.processor.DistributedZkUpdateProcessor.processAdd(DistributedZkUpdateProcessor.java:231)
at org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:103)
at org.apache.solr.update.UpdateLog$LogReplayer.lambda$execute$1(UpdateLog.java:2025)
at org.apache.solr.util.OrderedExecutor.lambda$execute$0(OrderedExecutor.java:69)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.update.TestInPlaceUpdatesDistrib
Error Message:
Captured an uncaught exception in thread: Thread[id=119830, name=replayUpdatesExecutor-20270-thread-1, state=RUNNABLE, group=TGRP-TestInPlaceUpdatesDistrib]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=119830, name=replayUpdatesExecutor-20270-thread-1, state=RUNNABLE, group=TGRP-TestInPlaceUpdatesDistrib]
Caused by: org.apache.solr.update.SolrCoreState$CoreIsClosedException
at __randomizedtesting.SeedInfo.seed([E3F3881478E51589]:0)
at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:2026)
at org.apache.solr.core.SolrCore.getRealtimeSearcher(SolrCore.java:2001)
at org.apache.solr.update.VersionInfo.getVersionFromIndex(VersionInfo.java:214)
at org.apache.solr.update.UpdateLog.lookupVersion(UpdateLog.java:1058)
at org.apache.solr.update.VersionInfo.lookupVersion(VersionInfo.java:203)
at org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:453)
at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:220)
at org.apache.solr.update.processor.DistributedZkUpdateProcessor.processAdd(DistributedZkUpdateProcessor.java:231)
at org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:103)
at org.apache.solr.update.UpdateLog$LogReplayer.lambda$execute$1(UpdateLog.java:2025)
at org.apache.solr.util.OrderedExecutor.lambda$execute$0(OrderedExecutor.java:69)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.SaslZkACLProviderTest
Error Message:
3 threads leaked from SUITE scope at org.apache.solr.cloud.SaslZkACLProviderTest: 1) Thread[id=162347, name=TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[E3F3881478E51589]-EventThread, state=WAITING, group=TGRP-SaslZkACLProviderTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:504) 2) Thread[id=162351, name=TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[E3F3881478E51589]-EventThread, state=WAITING, group=TGRP-SaslZkACLProviderTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:504) 3) Thread[id=162354, name=TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[E3F3881478E51589]-EventThread, state=WAITING, group=TGRP-SaslZkACLProviderTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:504)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 3 threads leaked from SUITE scope at org.apache.solr.cloud.SaslZkACLProviderTest:
1) Thread[id=162347, name=TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[E3F3881478E51589]-EventThread, state=WAITING, group=TGRP-SaslZkACLProviderTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:504)
2) Thread[id=162351, name=TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[E3F3881478E51589]-EventThread, state=WAITING, group=TGRP-SaslZkACLProviderTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:504)
3) Thread[id=162354, name=TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[E3F3881478E51589]-EventThread, state=WAITING, group=TGRP-SaslZkACLProviderTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:504)
at __randomizedtesting.SeedInfo.seed([E3F3881478E51589]:0)
FAILED: org.apache.solr.cloud.SaslZkACLProviderTest.testSaslZkACLProvider
Error Message:
KeeperErrorCode = AuthFailed for /solr
Stack Trace:
org.apache.zookeeper.KeeperException$AuthFailedException: KeeperErrorCode = AuthFailed for /solr
at __randomizedtesting.SeedInfo.seed([E3F3881478E51589:E707DECCE1A1B687]:0)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:126)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:792)
at org.apache.solr.common.cloud.SolrZkClient.lambda$makePath$8(SolrZkClient.java:545)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71)
at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:544)
at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:436)
at org.apache.solr.cloud.SaslZkACLProviderTest.setUp(SaslZkACLProviderTest.java:84)
at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:972)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 15086 lines...]
[junit4] Suite: org.apache.solr.cloud.SaslZkACLProviderTest
[junit4] 2> 5080512 INFO (SUITE-SaslZkACLProviderTest-seed#[E3F3881478E51589]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.SaslZkACLProviderTest_E3F3881478E51589-001/init-core-data-001
[junit4] 2> 5080513 WARN (SUITE-SaslZkACLProviderTest-seed#[E3F3881478E51589]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
[junit4] 2> 5080513 INFO (SUITE-SaslZkACLProviderTest-seed#[E3F3881478E51589]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 5080529 INFO (SUITE-SaslZkACLProviderTest-seed#[E3F3881478E51589]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 5080568 INFO (TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[E3F3881478E51589]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testSaslZkACLProvider
[junit4] 2> 5080568 INFO (TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[E3F3881478E51589]) [ ] o.a.s.c.SaslZkACLProviderTest ####SETUP_START testSaslZkACLProvider
[junit4] 2> 5080569 INFO (TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[E3F3881478E51589]) [ ] o.a.s.c.SaslZkACLProviderTest ZooKeeper dataDir:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.SaslZkACLProviderTest_E3F3881478E51589-001/tempDir-002/zookeeper/server1/data
[junit4] 2> 5080732 INFO (TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[E3F3881478E51589]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 5080750 INFO (ZkTestServer Run Thread) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 5080750 INFO (ZkTestServer Run Thread) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 5080820 INFO (pool-72-thread-1) [ ] o.a.k.k.k.s.r.KdcRequest The preauth data is empty.
[junit4] 2> 5080820 INFO (pool-72-thread-1) [ ] o.a.k.k.k.s.KdcHandler KRB error occurred while processing request:Additional pre-authentication required
[junit4] 2> 5080878 INFO (pool-72-thread-1) [ ] o.a.k.k.k.s.r.AsRequest AS_REQ ISSUE: authtime 1554373592627,zookeeper/localhost@EXAMPLE.COM for krbtgt/EXAMPLE.COM@EXAMPLE.COM
[junit4] 2> 5080969 INFO (TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[E3F3881478E51589]) [ ] o.a.s.c.ZkTestServer start zk server on port:41632
[junit4] 2> 5080969 INFO (TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[E3F3881478E51589]) [ ] o.a.s.c.ZkTestServer parse host and port list: localhost:41632
[junit4] 2> 5080969 INFO (TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[E3F3881478E51589]) [ ] o.a.s.c.ZkTestServer connecting to localhost 41632
[junit4] 2> 5081127 INFO (pool-72-thread-1) [ ] o.a.k.k.k.s.r.KdcRequest The preauth data is empty.
[junit4] 2> 5081127 INFO (pool-72-thread-1) [ ] o.a.k.k.k.s.KdcHandler KRB error occurred while processing request:Additional pre-authentication required
[junit4] 2> 5081277 INFO (pool-72-thread-1) [ ] o.a.k.k.k.s.r.AsRequest AS_REQ ISSUE: authtime 1554373593026,solr@EXAMPLE.COM for krbtgt/EXAMPLE.COM@EXAMPLE.COM
[junit4] 2> 5081460 INFO (zkConnectionManagerCallback-53227-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 5081461 ERROR (pool-72-thread-1) [ ] o.a.k.k.k.s.r.KdcRequest Principal: zookeeper/lucene2-us-west.apache.org@EXAMPLE.COM is not known
[junit4] 2> 5081462 ERROR (TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[E3F3881478E51589]-SendThread(localhost:41632)) [ ] o.a.z.c.ZooKeeperSaslClient An error: (java.security.PrivilegedActionException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7) - Server not found in Kerberos database)]) occurred when evaluating Zookeeper Quorum Member's received SASL token. Zookeeper Client will go to AUTH_FAILED state.
[junit4] 2> 5081462 ERROR (TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[E3F3881478E51589]-SendThread(localhost:41632)) [ ] o.a.z.ClientCnxn SASL authentication with Zookeeper Quorum member failed: javax.security.sasl.SaslException: An error: (java.security.PrivilegedActionException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7) - Server not found in Kerberos database)]) occurred when evaluating Zookeeper Quorum Member's received SASL token. Zookeeper Client will go to AUTH_FAILED state.
[junit4] 2> 5081550 WARN (zkConnectionManagerCallback-53227-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@ae1c5de name: ZooKeeperConnection Watcher:localhost:41632 got event WatchedEvent state:AuthFailed type:None path:null path: null type: None
[junit4] 2> 5081550 WARN (zkConnectionManagerCallback-53227-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient received AuthFailed
[junit4] 2> 5081641 INFO (zkConnectionManagerCallback-53229-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 5081695 INFO (zkConnectionManagerCallback-53231-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 5081803 ERROR (pool-72-thread-1) [ ] o.a.k.k.k.s.r.KdcRequest Principal: zookeeper/lucene2-us-west.apache.org@EXAMPLE.COM is not known
[junit4] 2> 5081804 ERROR (TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[E3F3881478E51589]-SendThread(localhost:41632)) [ ] o.a.z.c.ZooKeeperSaslClient An error: (java.security.PrivilegedActionException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7) - Server not found in Kerberos database)]) occurred when evaluating Zookeeper Quorum Member's received SASL token. Zookeeper Client will go to AUTH_FAILED state.
[junit4] 2> 5081804 ERROR (TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[E3F3881478E51589]-SendThread(localhost:41632)) [ ] o.a.z.ClientCnxn SASL authentication with Zookeeper Quorum member failed: javax.security.sasl.SaslException: An error: (java.security.PrivilegedActionException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7) - Server not found in Kerberos database)]) occurred when evaluating Zookeeper Quorum Member's received SASL token. Zookeeper Client will go to AUTH_FAILED state.
[junit4] 2> 5081805 WARN (zkConnectionManagerCallback-53229-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@464579fb name: ZooKeeperConnection Watcher:localhost:41632/solr got event WatchedEvent state:AuthFailed type:None path:null path: null type: None
[junit4] 2> 5081805 WARN (zkConnectionManagerCallback-53229-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient received AuthFailed
[junit4] 2> 5081806 ERROR (pool-72-thread-1) [ ] o.a.k.k.k.s.r.KdcRequest Principal: zookeeper/lucene2-us-west.apache.org@EXAMPLE.COM is not known
[junit4] 2> 5081807 ERROR (TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[E3F3881478E51589]-SendThread(localhost:41632)) [ ] o.a.z.c.ZooKeeperSaslClient An error: (java.security.PrivilegedActionException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7) - Server not found in Kerberos database)]) occurred when evaluating Zookeeper Quorum Member's received SASL token. Zookeeper Client will go to AUTH_FAILED state.
[junit4] 2> 5081807 ERROR (TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[E3F3881478E51589]-SendThread(localhost:41632)) [ ] o.a.z.ClientCnxn SASL authentication with Zookeeper Quorum member failed: javax.security.sasl.SaslException: An error: (java.security.PrivilegedActionException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7) - Server not found in Kerberos database)]) occurred when evaluating Zookeeper Quorum Member's received SASL token. Zookeeper Client will go to AUTH_FAILED state.
[junit4] 2> 5081820 WARN (zkConnectionManagerCallback-53231-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@53789a6f name: ZooKeeperConnection Watcher:localhost:41632 got event WatchedEvent state:AuthFailed type:None path:null path: null type: None
[junit4] 2> 5081820 WARN (zkConnectionManagerCallback-53231-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient received AuthFailed
[junit4] 2> 5081823 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1088c254ef70002, likely client has closed socket
[junit4] 2> 5082044 INFO (TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[E3F3881478E51589]) [ ] o.a.s.c.ZkTestServer Shutting down ZkTestServer.
[junit4] 2> 5082045 WARN (Thread-58749) [ ] o.a.z.Login TGT renewal thread has been interrupted and will exit.
[junit4] 2> 5082099 INFO (TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[E3F3881478E51589]) [ ] o.a.s.c.ZkTestServer parse host and port list: localhost:41632
[junit4] 2> 5082099 INFO (TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[E3F3881478E51589]) [ ] o.a.s.c.ZkTestServer connecting to localhost 41632
[junit4] 2> 5082100 INFO (TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[E3F3881478E51589]) [ ] o.a.k.k.k.s.i.DefaultInternalKdcServerImpl Default Internal kdc server stopped.
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=SaslZkACLProviderTest -Dtests.method=testSaslZkACLProvider -Dtests.seed=E3F3881478E51589 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/test-data/enwiki.random.lines.txt -Dtests.locale=bg-BG -Dtests.timezone=Asia/Manila -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] ERROR 2.58s J1 | SaslZkACLProviderTest.testSaslZkACLProvider <<<
[junit4] > Throwable #1: org.apache.zookeeper.KeeperException$AuthFailedException: KeeperErrorCode = AuthFailed for /solr
[junit4] > at __randomizedtesting.SeedInfo.seed([E3F3881478E51589:E707DECCE1A1B687]:0)
[junit4] > at org.apache.zookeeper.KeeperException.create(KeeperException.java:126)
[junit4] > at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
[junit4] > at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:792)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.lambda$makePath$8(SolrZkClient.java:545)
[junit4] > at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:71)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:544)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:436)
[junit4] > at org.apache.solr.cloud.SaslZkACLProviderTest.setUp(SaslZkACLProviderTest.java:84)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 5083103 INFO (TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[E3F3881478E51589]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testSaslZkACLProvider
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.SaslZkACLProviderTest_E3F3881478E51589-001
[junit4] 2> Apr 04, 2019 10:26:35 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 3 leaked thread(s).
[junit4] 2> Apr 04, 2019 10:26:45 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 3 threads leaked from SUITE scope at org.apache.solr.cloud.SaslZkACLProviderTest:
[junit4] 2> 1) Thread[id=162347, name=TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[E3F3881478E51589]-EventThread, state=WAITING, group=TGRP-SaslZkACLProviderTest]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:504)
[junit4] 2> 2) Thread[id=162351, name=TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[E3F3881478E51589]-EventThread, state=WAITING, group=TGRP-SaslZkACLProviderTest]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:504)
[junit4] 2> 3) Thread[id=162354, name=TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[E3F3881478E51589]-EventThread, state=WAITING, group=TGRP-SaslZkACLProviderTest]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:504)
[junit4] 2> Apr 04, 2019 10:26:45 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=162347, name=TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[E3F3881478E51589]-EventThread, state=WAITING, group=TGRP-SaslZkACLProviderTest]
[junit4] 2> 2) Thread[id=162351, name=TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[E3F3881478E51589]-EventThread, state=WAITING, group=TGRP-SaslZkACLProviderTest]
[junit4] 2> 3) Thread[id=162354, name=TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[E3F3881478E51589]-EventThread, state=WAITING, group=TGRP-SaslZkACLProviderTest]
[junit4] 2> 5093673 ERROR (TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[E3F3881478E51589]-EventThread) [ ] o.a.z.ClientCnxn Event thread exiting due to interruption
[junit4] 2> => java.lang.InterruptedException
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014)
[junit4] 2> java.lang.InterruptedException: null
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) ~[?:1.8.0_191]
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048) ~[?:1.8.0_191]
[junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) ~[?:1.8.0_191]
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:504) ~[zookeeper-3.4.14.jar:3.4.14-4c25d480e66aadd371de8bd2fd8da255ac140bcf]
[junit4] 2> 5093677 ERROR (TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[E3F3881478E51589]-EventThread) [ ] o.a.z.ClientCnxn Event thread exiting due to interruption
[junit4] 2> => java.lang.InterruptedException
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014)
[junit4] 2> java.lang.InterruptedException: null
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) ~[?:1.8.0_191]
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048) ~[?:1.8.0_191]
[junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) ~[?:1.8.0_191]
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:504) ~[zookeeper-3.4.14.jar:3.4.14-4c25d480e66aadd371de8bd2fd8da255ac140bcf]
[junit4] 2> 5093677 ERROR (TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[E3F3881478E51589]-EventThread) [ ] o.a.z.ClientCnxn Event thread exiting due to interruption
[junit4] 2> => java.lang.InterruptedException
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014)
[junit4] 2> java.lang.InterruptedException: null
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) ~[?:1.8.0_191]
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048) ~[?:1.8.0_191]
[junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) ~[?:1.8.0_191]
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:504) ~[zookeeper-3.4.14.jar:3.4.14-4c25d480e66aadd371de8bd2fd8da255ac140bcf]
[junit4] 2> Apr 04, 2019 10:26:45 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: All leaked threads terminated.
[junit4] 2> NOTE: test params are: codec=CheapBastard, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@390ced3b), locale=bg-BG, timezone=Asia/Manila
[junit4] 2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_191 (64-bit)/cpus=4,threads=2,free=210388208,total=524288000
[junit4] 2> NOTE: All tests run in this JVM: [ZkShardTermsTest, MetricsConfigTest, RequestHandlersTest, CloudExitableDirectoryReaderTest, DeleteShardTest, TestSegmentSorting, RecoveryZkTest, TestExceedMaxTermLength, TestTolerantUpdateProcessorRandomCloud, SolrRrdBackendFactoryTest, SpatialHeatmapFacetsTest, TestLRUStatsCache, NumericFieldsTest, ScheduledTriggerTest, CachingDirectoryFactoryTest, PreAnalyzedFieldManagedSchemaCloudTest, TestCopyFieldCollectionResource, TestCloudPhrasesIdentificationComponent, RuleEngineTest, DistributedUpdateProcessorTest, TestUnInvertedFieldException, TestNumericTokenStream, TestSolrCLIRunExample, TestFieldCacheSort, SyncSliceTest, TestMacros, TestCloudRecovery, SolrCloudExampleTest, TestConfigSetsAPIZkFailure, HttpPartitionOnCommitTest, BaseCdcrDistributedZkTest, TestCloudNestedDocsSort, Tagger2Test, SubstringBytesRefFilterTest, TestObjectReleaseTracker, BlobRepositoryCloudTest, SolrCoreCheckLockOnStartupTest, ScriptEngineTest, OrderedExecutorTest, CollectionPropsTest, AliasIntegrationTest, ZookeeperStatusHandlerTest, V2ApiIntegrationTest, TestBlendedInfixSuggestions, SolrInfoBeanTest, TestCollectionAPI, TestBinaryResponseWriter, CSVRequestHandlerTest, MultiThreadedOCPTest, TestShardHandlerFactory, TestDistributedStatsComponentCardinality, SharedFSAutoReplicaFailoverTest, LeaderElectionContextKeyTest, MetricTriggerTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, TestManagedResourceStorage, TestUnifiedSolrHighlighter, TestRTGBase, TestBlobHandler, TestReRankQParserPlugin, TestOverriddenPrefixQueryForCustomFieldType, TestScoreJoinQPScore, SimpleMLTQParserTest, CustomTermsComponentTest, ReplaceNodeNoTargetTest, SpatialFilterTest, DistanceFunctionTest, DistributedFacetPivotWhiteBoxTest, SolrGraphiteReporterTest, TestHalfAndHalfDocValues, CdcrVersionReplicationTest, TestLegacyFieldReuse, TestCollectionAPIs, MinimalSchemaTest, HdfsDirectoryTest, PluginInfoTest, TestFuzzyAnalyzedSuggestions, TestTolerantSearch, AddSchemaFieldsUpdateProcessorFactoryTest, TestFieldCollectionResource, TestPHPSerializedResponseWriter, DisMaxRequestHandlerTest, ComputePlanActionTest, TestSolrCloudWithHadoopAuthPlugin, TestFieldCacheWithThreads, CoreAdminRequestStatusTest, TestJsonFacetRefinement, TestSearchPerf, TestDistributedSearch, StatelessScriptUpdateProcessorFactoryTest, ResponseHeaderTest, RestoreTriggerStateTest, CursorMarkTest, TestBadConfig, HdfsChaosMonkeySafeLeaderTest, TestMultiValuedNumericRangeQuery, ConnectionReuseTest, TestSchemaManager, TestSha256AuthenticationProvider, BlockJoinFacetRandomTest, TestStressLiveNodes, TestSmileRequest, SpellCheckCollatorTest, TestRawTransformer, CdcrOpsAndBoundariesTest, DirectSolrSpellCheckerTest, SystemLogListenerTest, TestRecoveryHdfs, TestGraphTermsQParserPlugin, ClassificationUpdateProcessorTest, PeerSyncWithLeaderAndIndexFingerprintCachingTest, TestLocalFSCloudBackupRestore, CircularListTest, RequestLoggingTest, TestExclusionRuleCollectionAccess, TestNamedUpdateProcessors, TestReplicationHandlerBackup, ReturnFieldsTest, TestDocTermOrdsUninvertLimit, BasicDistributedZk2Test, ZkStateReaderTest, TestLegacyNumericRangeQueryBuilder, TestFieldTypeResource, ZkControllerTest, BigEndianAscendingWordSerializerTest, ShardsWhitelistTest, DirectoryFactoryTest, CreateCollectionCleanupTest, DeleteLastCustomShardedReplicaTest, TestJsonFacetsWithNestedObjects, TestNonDefinedSimilarityFactory, TestStressUserVersions, TestLMDirichletSimilarityFactory, SaslZkACLProviderTest]
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=SaslZkACLProviderTest -Dtests.seed=E3F3881478E51589 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/test-data/enwiki.random.lines.txt -Dtests.locale=bg-BG -Dtests.timezone=Asia/Manila -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] ERROR 0.00s J1 | SaslZkACLProviderTest (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 3 threads leaked from SUITE scope at org.apache.solr.cloud.SaslZkACLProviderTest:
[junit4] > 1) Thread[id=162347, name=TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[E3F3881478E51589]-EventThread, state=WAITING, group=TGRP-SaslZkACLProviderTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:504)
[junit4] > 2) Thread[id=162351, name=TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[E3F3881478E51589]-EventThread, state=WAITING, group=TGRP-SaslZkACLProviderTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:504)
[junit4] > 3) Thread[id=162354, name=TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[E3F3881478E51589]-EventThread, state=WAITING, group=TGRP-SaslZkACLProviderTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:504)
[junit4] > at __randomizedtesting.SeedInfo.seed([E3F3881478E51589]:0)
[junit4] Completed [451/846 (1!)] on J1 in 13.61s, 1 test, 2 errors <<< FAILURES!
[...truncated 1581 lines...]
[junit4] Suite: org.apache.solr.update.TestInPlaceUpdatesDistrib
[junit4] 2> 9210955 INFO (SUITE-TestInPlaceUpdatesDistrib-seed#[E3F3881478E51589]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.update.TestInPlaceUpdatesDistrib_E3F3881478E51589-001/init-core-data-001
[junit4] 2> 9210955 WARN (SUITE-TestInPlaceUpdatesDistrib-seed#[E3F3881478E51589]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=43 numCloses=43
[junit4] 2> 9210955 INFO (SUITE-TestInPlaceUpdatesDistrib-seed#[E3F3881478E51589]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 9210957 INFO (SUITE-TestInPlaceUpdatesDistrib-seed#[E3F3881478E51589]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 9210957 INFO (SUITE-TestInPlaceUpdatesDistrib-seed#[E3F3881478E51589]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 2> 9210957 INFO (SUITE-TestInPlaceUpdatesDistrib-seed#[E3F3881478E51589]-worker) [ ] o.a.s.SolrTestCaseJ4 updateLog impl=solr.CdcrUpdateLog
[junit4] 2> 9210957 INFO (SUITE-TestInPlaceUpdatesDistrib-seed#[E3F3881478E51589]-worker) [ ] o.a.s.SolrTestCaseJ4 ####initCore
[junit4] 2> 9210957 INFO (SUITE-TestInPlaceUpdatesDistrib-seed#[E3F3881478E51589]-worker) [ ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/lib, /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
[junit4] 2> 9210979 INFO (SUITE-TestInPlaceUpdatesDistrib-seed#[E3F3881478E51589]-worker) [ ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.1.0
[junit4] 2> 9210988 INFO (SUITE-TestInPlaceUpdatesDistrib-seed#[E3F3881478E51589]-worker) [ ] o.a.s.s.IndexSchema [null] Schema name=inplace-updates
[junit4] 2> 9210992 INFO (SUITE-TestInPlaceUpdatesDistrib-seed#[E3F3881478E51589]-worker) [ ] o.a.s.s.IndexSchema Loaded schema inplace-updates/1.6 with uniqueid field id
[junit4] 2> 9211313 INFO (SUITE-TestInPlaceUpdatesDistrib-seed#[E3F3881478E51589]-worker) [ ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=false]
[junit4] 2> 9211316 WARN (SUITE-TestInPlaceUpdatesDistrib-seed#[E3F3881478E51589]-worker) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@1e89e7c9[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 9211319 WARN (SUITE-TestInPlaceUpdatesDistrib-seed#[E3F3881478E51589]-worker) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@5013d664[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 9211334 INFO (SUITE-TestInPlaceUpdatesDistrib-seed#[E3F3881478E51589]-worker) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 9211334 INFO (SUITE-TestInPlaceUpdatesDistrib-seed#[E3F3881478E51589]-worker) [ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 9211347 INFO (SUITE-TestInPlaceUpdatesDistrib-seed#[E3F3881478E51589]-worker) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@188abcb8
[junit4] 2> 9211354 INFO (SUITE-TestInPlaceUpdatesDistrib-seed#[E3F3881478E51589]-worker) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@188abcb8
[junit4] 2> 9211354 INFO (SUITE-TestInPlaceUpdatesDistrib-seed#[E3F3881478E51589]-worker) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@188abcb8
[junit4] 2> 9211355 INFO (coreLoadExecutor-20232-thread-1) [ x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/lib, /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
[junit4] 2> 9211378 INFO (coreLoadExecutor-20232-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.1.0
[junit4] 2> 9211388 INFO (coreLoadExecutor-20232-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=inplace-updates
[junit4] 2> 9211391 INFO (coreLoadExecutor-20232-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema inplace-updates/1.6 with uniqueid field id
[junit4] 2> 9211391 INFO (coreLoadExecutor-20232-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1, trusted=true
[junit4] 2> 9211392 INFO (coreLoadExecutor-20232-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@188abcb8
[junit4] 2> 9211392 INFO (coreLoadExecutor-20232-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.update.TestInPlaceUpdatesDistrib_E3F3881478E51589-001/init-core-data-001/]
[junit4] 2> 9211397 WARN (coreLoadExecutor-20232-thread-1) [ x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 9211429 INFO (coreLoadExecutor-20232-thread-1) [ x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog
[junit4] 2> 9211429 INFO (coreLoadExecutor-20232-thread-1) [ x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 9211431 INFO (coreLoadExecutor-20232-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 9211431 INFO (coreLoadExecutor-20232-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 9211433 INFO (coreLoadExecutor-20232-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@47a45297[collection1] main]
[junit4] 2> 9211434 WARN (coreLoadExecutor-20232-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage Cannot write to config directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf; switching to use InMemory storage instead.
[junit4] 2> 9211434 INFO (coreLoadExecutor-20232-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 9211434 INFO (coreLoadExecutor-20232-thread-1) [ x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1629883175479017472
[junit4] 2> 9211437 INFO (searcherExecutor-20233-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@47a45297[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 9211437 INFO (SUITE-TestInPlaceUpdatesDistrib-seed#[E3F3881478E51589]-worker) [ ] o.a.s.SolrTestCaseJ4 ####initCore end
[junit4] 2> 9211442 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 9211443 INFO (ZkTestServer Run Thread) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 9211443 INFO (ZkTestServer Run Thread) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 9211543 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.c.ZkTestServer start zk server on port:34377
[junit4] 2> 9211543 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:34377
[junit4] 2> 9211543 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 34377
[junit4] 2> 9211548 INFO (zkConnectionManagerCallback-26473-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 9211550 INFO (zkConnectionManagerCallback-26475-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 9211552 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 9211553 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/schema-inplace-updates.xml to /configs/conf1/schema.xml
[junit4] 2> 9211554 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 9211554 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 9211555 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 9211556 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 9211557 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 9211557 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 9211558 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 9211559 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 9211560 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 9211561 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
[junit4] 2> 9211680 WARN (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.e.j.s.AbstractConnector Ignoring deprecated socket close linger time
[junit4] 2> 9211680 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.c.s.e.JettySolrRunner Start Jetty (original configured port=0)
[junit4] 2> 9211680 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
[junit4] 2> 9211680 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.e.j.s.Server jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: c4550056e785fb5665914545889f21dc136ad9e6; jvm 1.8.0_191-b12
[junit4] 2> 9211681 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 9211681 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 9211681 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 9211681 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@e55d2fd{/,null,AVAILABLE}
[junit4] 2> 9211682 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1a704c5a{HTTP/1.1,[http/1.1, h2c]}{127.0.0.1:41663}
[junit4] 2> 9211682 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.e.j.s.Server Started @9212273ms
[junit4] 2> 9211682 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.update.TestInPlaceUpdatesDistrib_E3F3881478E51589-001/tempDir-001/control/data, hostContext=/, hostPort=41663, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.update.TestInPlaceUpdatesDistrib_E3F3881478E51589-001/control-001/cores}
[junit4] 2> 9211682 ERROR (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 9211682 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 9211682 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.1.0
[junit4] 2> 9211682 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 9211682 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 9211682 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2019-04-04T11:35:23.445Z
[junit4] 2> 9211684 INFO (zkConnectionManagerCallback-26477-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 9211685 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 9211685 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.update.TestInPlaceUpdatesDistrib_E3F3881478E51589-001/control-001/solr.xml
[junit4] 2> 9211689 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 9211689 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 9211690 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@188abcb8, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 9211823 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=false]
[junit4] 2> 9211826 WARN (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@7c7d5d4c[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 9211830 WARN (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@7d11dbaa[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 9211832 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34377/solr
[junit4] 2> 9211833 INFO (zkConnectionManagerCallback-26484-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 9211836 INFO (zkConnectionManagerCallback-26486-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 9211967 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:41663_
[junit4] 2> 9211968 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.c.Overseer Overseer (id=74463756413829124-127.0.0.1:41663_-n_0000000000) starting
[junit4] 2> 9211996 INFO (zkConnectionManagerCallback-26493-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 9212000 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34377/solr ready
[junit4] 2> 9212002 INFO (OverseerStateUpdate-74463756413829124-127.0.0.1:41663_-n_0000000000) [n:127.0.0.1:41663_ ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:41663_
[junit4] 2> 9212002 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41663_
[junit4] 2> 9212004 INFO (OverseerStateUpdate-74463756413829124-127.0.0.1:41663_-n_0000000000) [n:127.0.0.1:41663_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 9212028 INFO (zkCallback-26492-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 9212042 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 9212064 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@188abcb8
[junit4] 2> 9212080 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@188abcb8
[junit4] 2> 9212081 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@188abcb8
[junit4] 2> 9212082 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.update.TestInPlaceUpdatesDistrib_E3F3881478E51589-001/control-001/cores
[junit4] 2> 9212171 INFO (zkConnectionManagerCallback-26499-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 9212176 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 9212178 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34377/solr ready
[junit4] 2> 9212180 INFO (qtp120441791-119574) [n:127.0.0.1:41663_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:41663_&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 9212186 INFO (OverseerThreadFactory-20247-thread-1) [ ] o.a.s.c.a.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 9212298 INFO (qtp120441791-119579) [n:127.0.0.1:41663_ x:control_collection_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 9212298 INFO (qtp120441791-119579) [n:127.0.0.1:41663_ x:control_collection_shard1_replica_n1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 9213313 INFO (qtp120441791-119579) [n:127.0.0.1:41663_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.1.0
[junit4] 2> 9213323 INFO (qtp120441791-119579) [n:127.0.0.1:41663_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=inplace-updates
[junit4] 2> 9213326 INFO (qtp120441791-119579) [n:127.0.0.1:41663_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema inplace-updates/1.6 with uniqueid field id
[junit4] 2> 9213326 INFO (qtp120441791-119579) [n:127.0.0.1:41663_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
[junit4] 2> 9213327 INFO (qtp120441791-119579) [n:127.0.0.1:41663_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@188abcb8
[junit4] 2> 9213327 INFO (qtp120441791-119579) [n:127.0.0.1:41663_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.update.TestInPlaceUpdatesDistrib_E3F3881478E51589-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.update.TestInPlaceUpdatesDistrib_E3F3881478E51589-001/control-001/cores/control_collection_shard1_replica_n1/data/]
[junit4] 2> 9213332 WARN (qtp120441791-119579) [n:127.0.0.1:41663_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 9213369 INFO (qtp120441791-119579) [n:127.0.0.1:41663_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog
[junit4] 2> 9213369 INFO (qtp120441791-119579) [n:127.0.0.1:41663_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 9213371 INFO (qtp120441791-119579) [n:127.0.0.1:41663_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 9213371 INFO (qtp120441791-119579) [n:127.0.0.1:41663_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 9213372 INFO (qtp120441791-119579) [n:127.0.0.1:41663_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4fce3750[control_collection_shard1_replica_n1] main]
[junit4] 2> 9213373 INFO (qtp120441791-119579) [n:127.0.0.1:41663_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 9213374 INFO (qtp120441791-119579) [n:127.0.0.1:41663_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 9213374 INFO (qtp120441791-119579) [n:127.0.0.1:41663_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 9213374 INFO (qtp120441791-119579) [n:127.0.0.1:41663_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1629883177513254912
[junit4] 2> 9213377 INFO (searcherExecutor-20252-thread-1-processing-n:127.0.0.1:41663_ x:control_collection_shard1_replica_n1 c:control_collection s:shard1) [n:127.0.0.1:41663_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@4fce3750[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 9213379 INFO (qtp120441791-119579) [n:127.0.0.1:41663_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/control_collection/terms/shard1 to Terms{values={core_node2=0}, version=0}
[junit4] 2> 9213379 INFO (qtp120441791-119579) [n:127.0.0.1:41663_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/control_collection/leaders/shard1
[junit4] 2> 9213382 INFO (qtp120441791-119579) [n:127.0.0.1:41663_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 9213382 INFO (qtp120441791-119579) [n:127.0.0.1:41663_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 9213382 INFO (qtp120441791-119579) [n:127.0.0.1:41663_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:41663/control_collection_shard1_replica_n1/
[junit4] 2> 9213382 INFO (qtp120441791-119579) [n:127.0.0.1:41663_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 9213382 INFO (qtp120441791-119579) [n:127.0.0.1:41663_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:41663/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 9213382 INFO (qtp120441791-119579) [n:127.0.0.1:41663_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/control_collection/leaders/shard1/leader after winning as /collections/control_collection/leader_elect/shard1/election/74463756413829124-core_node2-n_0000000000
[junit4] 2> 9213384 INFO (qtp120441791-119579) [n:127.0.0.1:41663_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:41663/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 9213485 INFO (zkCallback-26485-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 9213485 INFO (zkCallback-26485-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 9213487 INFO (qtp120441791-119579) [n:127.0.0.1:41663_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 9213490 INFO (qtp120441791-119579) [n:127.0.0.1:41663_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1192
[junit4] 2> 9213491 INFO (qtp120441791-119574) [n:127.0.0.1:41663_ ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas
[junit4] 2> 9213590 INFO (zkCallback-26485-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 9213590 INFO (zkCallback-26485-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 9213590 INFO (zkCallback-26485-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 9213596 INFO (qtp120441791-119574) [n:127.0.0.1:41663_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:41663_&wt=javabin&version=2} status=0 QTime=1415
[junit4] 2> 9213608 INFO (zkConnectionManagerCallback-26505-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 9213610 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 9213611 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34377/solr ready
[junit4] 2> 9213611 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 9213612 INFO (qtp120441791-119579) [n:127.0.0.1:41663_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&stateFormat=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 9213615 INFO (OverseerCollectionConfigSetProcessor-74463756413829124-127.0.0.1:41663_-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 9213615 INFO (OverseerThreadFactory-20247-thread-2) [ ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 9213820 WARN (OverseerThreadFactory-20247-thread-2) [ ] o.a.s.c.a.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
[junit4] 2> 9213822 INFO (qtp120441791-119579) [n:127.0.0.1:41663_ ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas
[junit4] 2> 9213823 INFO (qtp120441791-119579) [n:127.0.0.1:41663_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&stateFormat=1&wt=javabin&version=2} status=0 QTime=211
[junit4] 2> 9213825 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Creating jetty instances pullReplicaCount=0 numOtherReplicas=3
[junit4] 2> 9213937 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.update.TestInPlaceUpdatesDistrib_E3F3881478E51589-001/shard-1-001 of type NRT
[junit4] 2> 9213937 WARN (closeThreadPool-26506-thread-1) [ ] o.e.j.s.AbstractConnector Ignoring deprecated socket close linger time
[junit4] 2> 9213938 INFO (closeThreadPool-26506-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Start Jetty (original configured port=0)
[junit4] 2> 9213938 INFO (closeThreadPool-26506-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
[junit4] 2> 9213938 INFO (closeThreadPool-26506-thread-1) [ ] o.e.j.s.Server jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: c4550056e785fb5665914545889f21dc136ad9e6; jvm 1.8.0_191-b12
[junit4] 2> 9213940 INFO (closeThreadPool-26506-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 9213940 INFO (closeThreadPool-26506-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 9213940 INFO (closeThreadPool-26506-thread-1) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 9213941 INFO (closeThreadPool-26506-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@92a7f4{/,null,AVAILABLE}
[junit4] 2> 9213941 INFO (closeThreadPool-26506-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@6b4499bf{HTTP/1.1,[http/1.1, h2c]}{127.0.0.1:40852}
[junit4] 2> 9213941 INFO (closeThreadPool-26506-thread-1) [ ] o.e.j.s.Server Started @9214532ms
[junit4] 2> 9213941 INFO (closeThreadPool-26506-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.update.TestInPlaceUpdatesDistrib_E3F3881478E51589-001/tempDir-001/jetty1, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/, hostPort=40852, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.update.TestInPlaceUpdatesDistrib_E3F3881478E51589-001/shard-1-001/cores}
[junit4] 2> 9213941 ERROR (closeThreadPool-26506-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 9213941 INFO (closeThreadPool-26506-thread-1) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 9213941 INFO (closeThreadPool-26506-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.1.0
[junit4] 2> 9213941 INFO (closeThreadPool-26506-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 9213941 INFO (closeThreadPool-26506-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 9213941 INFO (closeThreadPool-26506-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2019-04-04T11:35:25.704Z
[junit4] 2> 9213952 INFO (zkConnectionManagerCallback-26508-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 9213953 INFO (closeThreadPool-26506-thread-1) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 9213953 INFO (closeThreadPool-26506-thread-1) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.update.TestInPlaceUpdatesDistrib_E3F3881478E51589-001/shard-1-001/solr.xml
[junit4] 2> 9213956 INFO (closeThreadPool-26506-thread-1) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 9213956 INFO (closeThreadPool-26506-thread-1) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 9213957 INFO (closeThreadPool-26506-thread-1) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@188abcb8, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 9214054 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.update.TestInPlaceUpdatesDistrib_E3F3881478E51589-001/shard-2-001 of type NRT
[junit4] 2> 9214064 WARN (closeThreadPool-26506-thread-2) [ ] o.e.j.s.AbstractConnector Ignoring deprecated socket close linger time
[junit4] 2> 9214064 INFO (closeThreadPool-26506-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Start Jetty (original configured port=0)
[junit4] 2> 9214064 INFO (closeThreadPool-26506-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
[junit4] 2> 9214064 INFO (closeThreadPool-26506-thread-2) [ ] o.e.j.s.Server jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: c4550056e785fb5665914545889f21dc136ad9e6; jvm 1.8.0_191-b12
[junit4] 2> 9214065 INFO (closeThreadPool-26506-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 9214065 INFO (closeThreadPool-26506-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 9214065 INFO (closeThreadPool-26506-thread-2) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 9214065 INFO (closeThreadPool-26506-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@407ca3bc{/,null,AVAILABLE}
[junit4] 2> 9214066 INFO (closeThreadPool-26506-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@769a11a0{HTTP/1.1,[http/1.1, h2c]}{127.0.0.1:41696}
[junit4] 2> 9214066 INFO (closeThreadPool-26506-thread-2) [ ] o.e.j.s.Server Started @9214657ms
[junit4] 2> 9214066 INFO (closeThreadPool-26506-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.update.TestInPlaceUpdatesDistrib_E3F3881478E51589-001/tempDir-001/jetty2, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/, hostPort=41696, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/../../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.update.TestInPlaceUpdatesDistrib_E3F3881478E51589-001/shard-2-001/cores}
[junit4] 2> 9214066 ERROR (closeThreadPool-26506-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 9214066 INFO (closeThreadPool-26506-thread-2) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 9214066 INFO (closeThreadPool-26506-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.1.0
[junit4] 2> 9214066 INFO (closeThreadPool-26506-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 9214066 INFO (closeThreadPool-26506-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 9214066 INFO (closeThreadPool-26506-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2019-04-04T11:35:25.829Z
[junit4] 2> 9214096 INFO (zkConnectionManagerCallback-26511-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 9214097 INFO (closeThreadPool-26506-thread-2) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 9214097 INFO (closeThreadPool-26506-thread-2) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.update.TestInPlaceUpdatesDistrib_E3F3881478E51589-001/shard-2-001/solr.xml
[junit4] 2> 9214100 INFO (closeThreadPool-26506-thread-2) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 9214100 INFO (closeThreadPool-26506-thread-2) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 9214101 INFO (closeThreadPool-26506-thread-2) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@188abcb8, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 9214194 INFO (closeThreadPool-26506-thread-1) [ ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=false]
[junit4] 2> 9214197 WARN (closeThreadPool-26506-thread-1) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@5d2599a3[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 9214201 WARN (closeThreadPool-26506-thread-1) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@6fb5a9c0[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 9214202 INFO (closeThreadPool-26506-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34377/solr
[junit4] 2> 9214204 INFO (zkConnectionManagerCallback-26518-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 9214208 INFO (zkConnectionManagerCallback-26520-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 9214219 INFO (TEST-TestInPlaceUpdatesDistrib.test-seed#[E3F3881478E51589]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.update.TestInPlaceUpdatesDistrib_E3F3881478E51589-001/shard-3-001 of type NRT
[junit4] 2> 9214219 INFO (closeThreadPool-26506-thread-1) [n:127.0.0.1:40852_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 9214220 WARN (closeThreadPool-26506-thread-3) [ ] o.e.j.s.AbstractConnector Ignoring deprecated socket close linger time
[junit4] 2> 9214220 INFO (closeThreadPool-26506-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Start Jetty (original configured port=0)
[junit4] 2> 9214220 INFO (closeThreadPool-26506-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
[junit4] 2> 9214220 INFO (closeThreadPool-26506-thread-3) [ ] o.e.j.s.Server jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: c4550056e785fb5665914545889f21dc136ad9e6; jvm 1.8.0_191-b12
[junit4] 2> 9214224 INFO (closeThreadPool-26506-thread-1) [n:127.0.0.1:40852_ ] o.a.s.c.ZkController Publish node=127.0.0.1:40852_ as DOWN
[junit4] 2> 9214225 INFO (closeThreadPool-26506-thread-1) [n:127.0.0.1:40852_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 9214225 INFO (closeThreadPool-26506-thread-1) [n:127.0.0.1:40852_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40852_
[junit4] 2> 9214226 INFO (zkCallback-26492-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 9214226 INFO (zkCallback-26485-thread-3) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 9214244 INFO (zkCallback-26504-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 9214244 INFO (zkCallback-26519-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 9214246 INFO (closeThreadPool-26506-thread-3) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 9214246 INFO (closeThreadPool-26506-thread-3) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 9214246 INFO (closeThreadPool-26506-thread-3) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 9214247 INFO (closeThreadPool-26506-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5c1e3780{/,null,AVAILABLE}
[junit4] 2> 9214247 INFO (closeThreadPool-26506-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1de5a357{HTTP/1.1,[http/1.1, h2c]}{127.0.0.1:34789}
[junit4] 2> 9214247 INFO (closeThreadPool-26506-thread-3) [ ] o.e.j.s.Server Started @9214838ms
[junit4] 2> 9214247 INFO (closeThreadPool-26506-thread-3) [ ] o.a.s.c.s.e.
[...truncated too long message...]
adresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/null5803492
[copy] Copying 240 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/null5803492
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/null5803492
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 153ms :: artifacts dl 12ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 719 minutes 25 seconds
Archiving artifacts
java.lang.InterruptedException: no matches found within 10000
at hudson.FilePath$ValidateAntFileMask.hasMatch(FilePath.java:2847)
at hudson.FilePath$ValidateAntFileMask.invoke(FilePath.java:2726)
at hudson.FilePath$ValidateAntFileMask.invoke(FilePath.java:2707)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3086)
Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to lucene2
at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
at hudson.remoting.Channel.call(Channel.java:955)
at hudson.FilePath.act(FilePath.java:1072)
at hudson.FilePath.act(FilePath.java:1061)
at hudson.FilePath.validateAntFileMask(FilePath.java:2705)
at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
at hudson.model.Run.execute(Run.java:1835)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused: hudson.FilePath$TunneledInterruptedException
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3088)
at hudson.remoting.UserRequest.perform(UserRequest.java:212)
at hudson.remoting.UserRequest.perform(UserRequest.java:54)
at hudson.remoting.Request$2.run(Request.java:369)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Caused: java.lang.InterruptedException: java.lang.InterruptedException: no matches found within 10000
at hudson.FilePath.act(FilePath.java:1074)
at hudson.FilePath.act(FilePath.java:1061)
at hudson.FilePath.validateAntFileMask(FilePath.java:2705)
at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
at hudson.model.Run.execute(Run.java:1835)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
[JENKINS] Lucene-Solr-NightlyTests-8.x - Build # 66 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-8.x/66/
3 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeyNothingIsSafeWithPullReplicasTest
Error Message:
ObjectTracker found 4 object(s) that were not released!!! [InternalHttpClient, MMapDirectory, SolrCore, MMapDirectory] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:321) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:330) at org.apache.solr.handler.IndexFetcher.createHttpClient(IndexFetcher.java:230) at org.apache.solr.handler.IndexFetcher.<init>(IndexFetcher.java:272) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:420) at org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:237) at org.apache.solr.cloud.RecoveryStrategy.doReplicateOnlyRecovery(RecoveryStrategy.java:382) at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:328) at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:307) at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MMapDirectory at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:509) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:351) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:424) at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$13(ReplicationHandler.java:1193) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.core.SolrCore at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1063) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:883) at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1245) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1155) at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92) at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360) at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:396) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:180) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199) at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:784) at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:750) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:510) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:397) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:343) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:165) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:703) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:502) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364) at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:305) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MMapDirectory at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:509) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:351) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:424) at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$13(ReplicationHandler.java:1193) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) expected null, but was:<ObjectTracker found 4 object(s) that were not released!!! [InternalHttpClient, MMapDirectory, SolrCore, MMapDirectory] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:321) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:330) at org.apache.solr.handler.IndexFetcher.createHttpClient(IndexFetcher.java:230) at org.apache.solr.handler.IndexFetcher.<init>(IndexFetcher.java:272) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:420) at org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:237) at org.apache.solr.cloud.RecoveryStrategy.doReplicateOnlyRecovery(RecoveryStrategy.java:382) at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:328) at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:307) at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MMapDirectory at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:509) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:351) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:424) at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$13(ReplicationHandler.java:1193) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.core.SolrCore at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1063) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:883) at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1245) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1155) at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92) at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360) at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:396) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:180) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199) at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:784) at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:750) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:510) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:397) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:343) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:165) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:703) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:502) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364) at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:305) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MMapDirectory at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:509) at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:351) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:424) at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$13(ReplicationHandler.java:1193) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) >
Stack Trace:
java.lang.AssertionError: ObjectTracker found 4 object(s) that were not released!!! [InternalHttpClient, MMapDirectory, SolrCore, MMapDirectory]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:321)
at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:330)
at org.apache.solr.handler.IndexFetcher.createHttpClient(IndexFetcher.java:230)
at org.apache.solr.handler.IndexFetcher.<init>(IndexFetcher.java:272)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:420)
at org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:237)
at org.apache.solr.cloud.RecoveryStrategy.doReplicateOnlyRecovery(RecoveryStrategy.java:382)
at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:328)
at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:307)
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MMapDirectory
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:509)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:351)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:424)
at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$13(ReplicationHandler.java:1193)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.core.SolrCore
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1063)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:883)
at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1245)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1155)
at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92)
at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360)
at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:396)
at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:180)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:784)
at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:750)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:510)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:397)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:343)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:165)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:703)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:502)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)
at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:305)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
at java.lang.Thread.run(Thread.java:748)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MMapDirectory
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:509)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:351)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:424)
at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$13(ReplicationHandler.java:1193)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
expected null, but was:<ObjectTracker found 4 object(s) that were not released!!! [InternalHttpClient, MMapDirectory, SolrCore, MMapDirectory]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:321)
at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:330)
at org.apache.solr.handler.IndexFetcher.createHttpClient(IndexFetcher.java:230)
at org.apache.solr.handler.IndexFetcher.<init>(IndexFetcher.java:272)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:420)
at org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:237)
at org.apache.solr.cloud.RecoveryStrategy.doReplicateOnlyRecovery(RecoveryStrategy.java:382)
at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:328)
at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:307)
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MMapDirectory
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:509)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:351)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:424)
at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$13(ReplicationHandler.java:1193)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.core.SolrCore
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1063)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:883)
at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1245)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1155)
at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92)
at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360)
at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:396)
at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:180)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:784)
at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:750)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:510)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:397)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:343)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:165)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:703)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:502)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)
at org.eclipse.jetty.server.HttpChannel.run(HttpChannel.java:305)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
at java.lang.Thread.run(Thread.java:748)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MMapDirectory
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:509)
at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:351)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:424)
at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$13(ReplicationHandler.java:1193)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
>
at __randomizedtesting.SeedInfo.seed([C6E42D8DDF884DA6]:0)
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotNull(Assert.java:755)
at org.junit.Assert.assertNull(Assert.java:737)
at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:325)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:901)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.hdfs.StressHdfsTest.test
Error Message:
Timeout occurred while waiting response from server at: http://127.0.0.1:41870/gy_t/w/delete_data_dir
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occurred while waiting response from server at: http://127.0.0.1:41870/gy_t/w/delete_data_dir
at __randomizedtesting.SeedInfo.seed([C6E42D8DDF884DA6:4EB012577174205E]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:660)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:207)
at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:504)
at org.apache.solr.client.solrj.SolrClient.commit(SolrClient.java:479)
at org.apache.solr.cloud.hdfs.StressHdfsTest.createAndDeleteCollection(StressHdfsTest.java:189)
at org.apache.solr.cloud.hdfs.StressHdfsTest.test(StressHdfsTest.java:103)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1082)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:1054)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:548)
... 48 more
FAILED: org.apache.solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest.testSimple
Error Message:
Waiting for collection testSimple1 Timeout waiting to see state for collection=testSimple1 :DocCollection(testSimple1//collections/testSimple1/state.json/30)={ "pullReplicas":"0", "replicationFactor":"2", "shards":{ "shard1":{ "range":"80000000-ffffffff", "state":"active", "replicas":{ "core_node2":{ "dataDir":"hdfs://lucene2-us-west.apache.org:44155/solr_hdfs_home/testSimple1/core_node2/data/", "base_url":"http://127.0.0.1:38158/solr", "node_name":"127.0.0.1:38158_solr", "type":"NRT", "force_set_state":"false", "ulogDir":"hdfs://lucene2-us-west.apache.org:44155/solr_hdfs_home/testSimple1/core_node2/data/tlog", "core":"testSimple1_shard1_replica_n1", "shared_storage":"true", "state":"recovering"}, "core_node4":{ "dataDir":"hdfs://lucene2-us-west.apache.org:44155/solr_hdfs_home/testSimple1/core_node4/data/", "base_url":"http://127.0.0.1:38158/solr", "node_name":"127.0.0.1:38158_solr", "type":"NRT", "force_set_state":"false", "ulogDir":"hdfs://lucene2-us-west.apache.org:44155/solr_hdfs_home/testSimple1/core_node4/data/tlog", "core":"testSimple1_shard1_replica_n3", "shared_storage":"true", "state":"active", "leader":"true"}}}, "shard2":{ "range":"0-7fffffff", "state":"active", "replicas":{ "core_node6":{ "dataDir":"hdfs://lucene2-us-west.apache.org:44155/solr_hdfs_home/testSimple1/core_node6/data/", "base_url":"http://127.0.0.1:38158/solr", "node_name":"127.0.0.1:38158_solr", "type":"NRT", "force_set_state":"false", "ulogDir":"hdfs://lucene2-us-west.apache.org:44155/solr_hdfs_home/testSimple1/core_node6/data/tlog", "core":"testSimple1_shard2_replica_n5", "shared_storage":"true", "state":"active"}, "core_node8":{ "dataDir":"hdfs://lucene2-us-west.apache.org:44155/solr_hdfs_home/testSimple1/core_node8/data/", "base_url":"http://127.0.0.1:38158/solr", "node_name":"127.0.0.1:38158_solr", "type":"NRT", "force_set_state":"false", "ulogDir":"hdfs://lucene2-us-west.apache.org:44155/solr_hdfs_home/testSimple1/core_node8/data/tlog", "core":"testSimple1_shard2_replica_n7", "shared_storage":"true", "state":"active", "leader":"true"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"2", "autoAddReplicas":"true", "nrtReplicas":"2", "tlogReplicas":"0"} Live Nodes: [127.0.0.1:35179_solr, 127.0.0.1:38158_solr] Last available state: DocCollection(testSimple1//collections/testSimple1/state.json/30)={ "pullReplicas":"0", "replicationFactor":"2", "shards":{ "shard1":{ "range":"80000000-ffffffff", "state":"active", "replicas":{ "core_node2":{ "dataDir":"hdfs://lucene2-us-west.apache.org:44155/solr_hdfs_home/testSimple1/core_node2/data/", "base_url":"http://127.0.0.1:38158/solr", "node_name":"127.0.0.1:38158_solr", "type":"NRT", "force_set_state":"false", "ulogDir":"hdfs://lucene2-us-west.apache.org:44155/solr_hdfs_home/testSimple1/core_node2/data/tlog", "core":"testSimple1_shard1_replica_n1", "shared_storage":"true", "state":"recovering"}, "core_node4":{ "dataDir":"hdfs://lucene2-us-west.apache.org:44155/solr_hdfs_home/testSimple1/core_node4/data/", "base_url":"http://127.0.0.1:38158/solr", "node_name":"127.0.0.1:38158_solr", "type":"NRT", "force_set_state":"false", "ulogDir":"hdfs://lucene2-us-west.apache.org:44155/solr_hdfs_home/testSimple1/core_node4/data/tlog", "core":"testSimple1_shard1_replica_n3", "shared_storage":"true", "state":"active", "leader":"true"}}}, "shard2":{ "range":"0-7fffffff", "state":"active", "replicas":{ "core_node6":{ "dataDir":"hdfs://lucene2-us-west.apache.org:44155/solr_hdfs_home/testSimple1/core_node6/data/", "base_url":"http://127.0.0.1:38158/solr", "node_name":"127.0.0.1:38158_solr", "type":"NRT", "force_set_state":"false", "ulogDir":"hdfs://lucene2-us-west.apache.org:44155/solr_hdfs_home/testSimple1/core_node6/data/tlog", "core":"testSimple1_shard2_replica_n5", "shared_storage":"true", "state":"active"}, "core_node8":{ "dataDir":"hdfs://lucene2-us-west.apache.org:44155/solr_hdfs_home/testSimple1/core_node8/data/", "base_url":"http://127.0.0.1:38158/solr", "node_name":"127.0.0.1:38158_solr", "type":"NRT", "force_set_state":"false", "ulogDir":"hdfs://lucene2-us-west.apache.org:44155/solr_hdfs_home/testSimple1/core_node8/data/tlog", "core":"testSimple1_shard2_replica_n7", "shared_storage":"true", "state":"active", "leader":"true"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"2", "autoAddReplicas":"true", "nrtReplicas":"2", "tlogReplicas":"0"}
Stack Trace:
java.lang.AssertionError: Waiting for collection testSimple1
Timeout waiting to see state for collection=testSimple1 :DocCollection(testSimple1//collections/testSimple1/state.json/30)={
"pullReplicas":"0",
"replicationFactor":"2",
"shards":{
"shard1":{
"range":"80000000-ffffffff",
"state":"active",
"replicas":{
"core_node2":{
"dataDir":"hdfs://lucene2-us-west.apache.org:44155/solr_hdfs_home/testSimple1/core_node2/data/",
"base_url":"http://127.0.0.1:38158/solr",
"node_name":"127.0.0.1:38158_solr",
"type":"NRT",
"force_set_state":"false",
"ulogDir":"hdfs://lucene2-us-west.apache.org:44155/solr_hdfs_home/testSimple1/core_node2/data/tlog",
"core":"testSimple1_shard1_replica_n1",
"shared_storage":"true",
"state":"recovering"},
"core_node4":{
"dataDir":"hdfs://lucene2-us-west.apache.org:44155/solr_hdfs_home/testSimple1/core_node4/data/",
"base_url":"http://127.0.0.1:38158/solr",
"node_name":"127.0.0.1:38158_solr",
"type":"NRT",
"force_set_state":"false",
"ulogDir":"hdfs://lucene2-us-west.apache.org:44155/solr_hdfs_home/testSimple1/core_node4/data/tlog",
"core":"testSimple1_shard1_replica_n3",
"shared_storage":"true",
"state":"active",
"leader":"true"}}},
"shard2":{
"range":"0-7fffffff",
"state":"active",
"replicas":{
"core_node6":{
"dataDir":"hdfs://lucene2-us-west.apache.org:44155/solr_hdfs_home/testSimple1/core_node6/data/",
"base_url":"http://127.0.0.1:38158/solr",
"node_name":"127.0.0.1:38158_solr",
"type":"NRT",
"force_set_state":"false",
"ulogDir":"hdfs://lucene2-us-west.apache.org:44155/solr_hdfs_home/testSimple1/core_node6/data/tlog",
"core":"testSimple1_shard2_replica_n5",
"shared_storage":"true",
"state":"active"},
"core_node8":{
"dataDir":"hdfs://lucene2-us-west.apache.org:44155/solr_hdfs_home/testSimple1/core_node8/data/",
"base_url":"http://127.0.0.1:38158/solr",
"node_name":"127.0.0.1:38158_solr",
"type":"NRT",
"force_set_state":"false",
"ulogDir":"hdfs://lucene2-us-west.apache.org:44155/solr_hdfs_home/testSimple1/core_node8/data/tlog",
"core":"testSimple1_shard2_replica_n7",
"shared_storage":"true",
"state":"active",
"leader":"true"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"2",
"autoAddReplicas":"true",
"nrtReplicas":"2",
"tlogReplicas":"0"}
Live Nodes: [127.0.0.1:35179_solr, 127.0.0.1:38158_solr]
Last available state: DocCollection(testSimple1//collections/testSimple1/state.json/30)={
"pullReplicas":"0",
"replicationFactor":"2",
"shards":{
"shard1":{
"range":"80000000-ffffffff",
"state":"active",
"replicas":{
"core_node2":{
"dataDir":"hdfs://lucene2-us-west.apache.org:44155/solr_hdfs_home/testSimple1/core_node2/data/",
"base_url":"http://127.0.0.1:38158/solr",
"node_name":"127.0.0.1:38158_solr",
"type":"NRT",
"force_set_state":"false",
"ulogDir":"hdfs://lucene2-us-west.apache.org:44155/solr_hdfs_home/testSimple1/core_node2/data/tlog",
"core":"testSimple1_shard1_replica_n1",
"shared_storage":"true",
"state":"recovering"},
"core_node4":{
"dataDir":"hdfs://lucene2-us-west.apache.org:44155/solr_hdfs_home/testSimple1/core_node4/data/",
"base_url":"http://127.0.0.1:38158/solr",
"node_name":"127.0.0.1:38158_solr",
"type":"NRT",
"force_set_state":"false",
"ulogDir":"hdfs://lucene2-us-west.apache.org:44155/solr_hdfs_home/testSimple1/core_node4/data/tlog",
"core":"testSimple1_shard1_replica_n3",
"shared_storage":"true",
"state":"active",
"leader":"true"}}},
"shard2":{
"range":"0-7fffffff",
"state":"active",
"replicas":{
"core_node6":{
"dataDir":"hdfs://lucene2-us-west.apache.org:44155/solr_hdfs_home/testSimple1/core_node6/data/",
"base_url":"http://127.0.0.1:38158/solr",
"node_name":"127.0.0.1:38158_solr",
"type":"NRT",
"force_set_state":"false",
"ulogDir":"hdfs://lucene2-us-west.apache.org:44155/solr_hdfs_home/testSimple1/core_node6/data/tlog",
"core":"testSimple1_shard2_replica_n5",
"shared_storage":"true",
"state":"active"},
"core_node8":{
"dataDir":"hdfs://lucene2-us-west.apache.org:44155/solr_hdfs_home/testSimple1/core_node8/data/",
"base_url":"http://127.0.0.1:38158/solr",
"node_name":"127.0.0.1:38158_solr",
"type":"NRT",
"force_set_state":"false",
"ulogDir":"hdfs://lucene2-us-west.apache.org:44155/solr_hdfs_home/testSimple1/core_node8/data/tlog",
"core":"testSimple1_shard2_replica_n7",
"shared_storage":"true",
"state":"active",
"leader":"true"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"2",
"autoAddReplicas":"true",
"nrtReplicas":"2",
"tlogReplicas":"0"}
at __randomizedtesting.SeedInfo.seed([C6E42D8DDF884DA6:FE570973F87B9977]:0)
at org.junit.Assert.fail(Assert.java:88)
at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:310)
at org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple(AutoAddReplicasIntegrationTest.java:168)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 13541 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest_C6E42D8DDF884DA6-001/init-core-data-001
[junit4] 2> 574835 WARN (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[C6E42D8DDF884DA6]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=99 numCloses=99
[junit4] 2> 574835 INFO (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[C6E42D8DDF884DA6]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 574837 INFO (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[C6E42D8DDF884DA6]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 574837 INFO (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[C6E42D8DDF884DA6]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 575395 WARN (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[C6E42D8DDF884DA6]-worker) [ ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 576996 WARN (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[C6E42D8DDF884DA6]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 577481 WARN (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[C6E42D8DDF884DA6]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 577574 INFO (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[C6E42D8DDF884DA6]-worker) [ ] o.e.j.s.Server jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: c4550056e785fb5665914545889f21dc136ad9e6; jvm 1.8.0_191-b12
[junit4] 2> 577578 INFO (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[C6E42D8DDF884DA6]-worker) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 577578 INFO (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[C6E42D8DDF884DA6]-worker) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 577578 INFO (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[C6E42D8DDF884DA6]-worker) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 577581 INFO (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[C6E42D8DDF884DA6]-worker) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3a9f1f53{static,/static,jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-3.2.0-tests.jar!/webapps/static,AVAILABLE}
[junit4] 2> 577922 INFO (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[C6E42D8DDF884DA6]-worker) [ ] o.e.j.s.h.ContextHandler Started o.e.j.w.WebAppContext@6f32318a{hdfs,/,file:///home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/jetty-lucene2-us-west.apache.org-34561-hdfs-_-any-123844435362201004.dir/webapp/,AVAILABLE}{/hdfs}
[junit4] 2> 577923 INFO (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[C6E42D8DDF884DA6]-worker) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2b261223{HTTP/1.1,[http/1.1]}{lucene2-us-west.apache.org:34561}
[junit4] 2> 577923 INFO (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[C6E42D8DDF884DA6]-worker) [ ] o.e.j.s.Server Started @578024ms
[junit4] 2> 580955 WARN (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[C6E42D8DDF884DA6]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 580969 INFO (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[C6E42D8DDF884DA6]-worker) [ ] o.e.j.s.Server jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: c4550056e785fb5665914545889f21dc136ad9e6; jvm 1.8.0_191-b12
[junit4] 2> 580993 INFO (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[C6E42D8DDF884DA6]-worker) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 580993 INFO (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[C6E42D8DDF884DA6]-worker) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 580993 INFO (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[C6E42D8DDF884DA6]-worker) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 580994 INFO (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[C6E42D8DDF884DA6]-worker) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@36678542{static,/static,jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-3.2.0-tests.jar!/webapps/static,AVAILABLE}
[junit4] 2> 581241 INFO (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[C6E42D8DDF884DA6]-worker) [ ] o.e.j.s.h.ContextHandler Started o.e.j.w.WebAppContext@5d92a9f1{datanode,/,file:///home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/jetty-localhost-37737-datanode-_-any-3084570906321665497.dir/webapp/,AVAILABLE}{/datanode}
[junit4] 2> 581248 INFO (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[C6E42D8DDF884DA6]-worker) [ ] o.e.j.s.AbstractConnector Started ServerConnector@48b54eb2{HTTP/1.1,[http/1.1]}{localhost:37737}
[junit4] 2> 581248 INFO (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[C6E42D8DDF884DA6]-worker) [ ] o.e.j.s.Server Started @581349ms
[junit4] 2> 583840 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0xe120ff84b4bf9b5f: Processing first storage report for DS-938c598b-9084-4f80-a18d-1a64e2e5f3a8 from datanode 7afd64d6-11ac-48ea-bf3a-ca055d8e5a75
[junit4] 2> 583843 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0xe120ff84b4bf9b5f: from storage DS-938c598b-9084-4f80-a18d-1a64e2e5f3a8 node DatanodeRegistration(127.0.0.1:44746, datanodeUuid=7afd64d6-11ac-48ea-bf3a-ca055d8e5a75, infoPort=39940, infoSecurePort=0, ipcPort=37826, storageInfo=lv=-57;cid=testClusterID;nsid=307120963;c=1554702041510), blocks: 0, hasStaleStorage: true, processing time: 3 msecs, invalidatedBlocks: 0
[junit4] 2> 583843 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0xe120ff84b4bf9b5f: Processing first storage report for DS-eba398ff-11a3-4de5-b61e-41472abf7e1e from datanode 7afd64d6-11ac-48ea-bf3a-ca055d8e5a75
[junit4] 2> 583843 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0xe120ff84b4bf9b5f: from storage DS-eba398ff-11a3-4de5-b61e-41472abf7e1e node DatanodeRegistration(127.0.0.1:44746, datanodeUuid=7afd64d6-11ac-48ea-bf3a-ca055d8e5a75, infoPort=39940, infoSecurePort=0, ipcPort=37826, storageInfo=lv=-57;cid=testClusterID;nsid=307120963;c=1554702041510), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0
[junit4] 2> 584300 INFO (TEST-HdfsAutoAddReplicasIntegrationTest.testSimple-seed#[C6E42D8DDF884DA6]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testSimple
[junit4] 2> 584301 INFO (TEST-HdfsAutoAddReplicasIntegrationTest.testSimple-seed#[C6E42D8DDF884DA6]) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 3 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest_C6E42D8DDF884DA6-001/tempDir-002
[junit4] 2> 584301 INFO (TEST-HdfsAutoAddReplicasIntegrationTest.testSimple-seed#[C6E42D8DDF884DA6]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 584301 INFO (ZkTestServer Run Thread) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 584301 INFO (ZkTestServer Run Thread) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 584401 INFO (TEST-HdfsAutoAddReplicasIntegrationTest.testSimple-seed#[C6E42D8DDF884DA6]) [ ] o.a.s.c.ZkTestServer start zk server on port:46295
[junit4] 2> 584401 INFO (TEST-HdfsAutoAddReplicasIntegrationTest.testSimple-seed#[C6E42D8DDF884DA6]) [ ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:46295
[junit4] 2> 584402 INFO (TEST-HdfsAutoAddReplicasIntegrationTest.testSimple-seed#[C6E42D8DDF884DA6]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 46295
[junit4] 2> 584419 INFO (zkConnectionManagerCallback-362-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 584459 INFO (zkConnectionManagerCallback-364-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 584500 INFO (zkConnectionManagerCallback-366-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 584513 WARN (jetty-launcher-367-thread-1) [ ] o.e.j.s.AbstractConnector Ignoring deprecated socket close linger time
[junit4] 2> 584513 INFO (jetty-launcher-367-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Start Jetty (original configured port=0)
[junit4] 2> 584514 INFO (jetty-launcher-367-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
[junit4] 2> 584514 INFO (jetty-launcher-367-thread-1) [ ] o.e.j.s.Server jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: c4550056e785fb5665914545889f21dc136ad9e6; jvm 1.8.0_191-b12
[junit4] 2> 584514 WARN (jetty-launcher-367-thread-2) [ ] o.e.j.s.AbstractConnector Ignoring deprecated socket close linger time
[junit4] 2> 584515 INFO (jetty-launcher-367-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Start Jetty (original configured port=0)
[junit4] 2> 584515 INFO (jetty-launcher-367-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
[junit4] 2> 584515 INFO (jetty-launcher-367-thread-2) [ ] o.e.j.s.Server jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: c4550056e785fb5665914545889f21dc136ad9e6; jvm 1.8.0_191-b12
[junit4] 2> 584516 INFO (jetty-launcher-367-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 584516 INFO (jetty-launcher-367-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 584516 INFO (jetty-launcher-367-thread-1) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 584516 INFO (jetty-launcher-367-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@74fc8e32{/solr,null,AVAILABLE}
[junit4] 2> 584518 INFO (jetty-launcher-367-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 584518 INFO (jetty-launcher-367-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 584518 INFO (jetty-launcher-367-thread-2) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 584518 INFO (jetty-launcher-367-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@650f6193{/solr,null,AVAILABLE}
[junit4] 2> 584540 WARN (jetty-launcher-367-thread-3) [ ] o.e.j.s.AbstractConnector Ignoring deprecated socket close linger time
[junit4] 2> 584540 INFO (jetty-launcher-367-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Start Jetty (original configured port=0)
[junit4] 2> 584540 INFO (jetty-launcher-367-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
[junit4] 2> 584540 INFO (jetty-launcher-367-thread-3) [ ] o.e.j.s.Server jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: c4550056e785fb5665914545889f21dc136ad9e6; jvm 1.8.0_191-b12
[junit4] 2> 584542 INFO (jetty-launcher-367-thread-3) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 584542 INFO (jetty-launcher-367-thread-3) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 584542 INFO (jetty-launcher-367-thread-3) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 584543 INFO (jetty-launcher-367-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@32bff58b{/solr,null,AVAILABLE}
[junit4] 2> 584552 INFO (jetty-launcher-367-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1775accc{HTTP/1.1,[http/1.1, h2c]}{127.0.0.1:40208}
[junit4] 2> 584552 INFO (jetty-launcher-367-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@497ff9bb{HTTP/1.1,[http/1.1, h2c]}{127.0.0.1:35179}
[junit4] 2> 584553 INFO (jetty-launcher-367-thread-1) [ ] o.e.j.s.Server Started @584654ms
[junit4] 2> 584553 INFO (jetty-launcher-367-thread-2) [ ] o.e.j.s.Server Started @584654ms
[junit4] 2> 584553 INFO (jetty-launcher-367-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40208}
[junit4] 2> 584553 INFO (jetty-launcher-367-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35179}
[junit4] 2> 584553 ERROR (jetty-launcher-367-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 584553 ERROR (jetty-launcher-367-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 584553 INFO (jetty-launcher-367-thread-1) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 584553 INFO (jetty-launcher-367-thread-2) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 584553 INFO (jetty-launcher-367-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.1.0
[junit4] 2> 584553 INFO (jetty-launcher-367-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.1.0
[junit4] 2> 584553 INFO (jetty-launcher-367-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 584553 INFO (jetty-launcher-367-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 584553 INFO (jetty-launcher-367-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 584553 INFO (jetty-launcher-367-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 584553 INFO (jetty-launcher-367-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2019-04-08T05:40:49.916Z
[junit4] 2> 584553 INFO (jetty-launcher-367-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2019-04-08T05:40:49.916Z
[junit4] 2> 584577 INFO (jetty-launcher-367-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@59bac2c2{HTTP/1.1,[http/1.1, h2c]}{127.0.0.1:38158}
[junit4] 2> 584577 INFO (jetty-launcher-367-thread-3) [ ] o.e.j.s.Server Started @584678ms
[junit4] 2> 584577 INFO (zkConnectionManagerCallback-369-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 584577 INFO (jetty-launcher-367-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=38158}
[junit4] 2> 584577 ERROR (jetty-launcher-367-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 584577 INFO (jetty-launcher-367-thread-3) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 584577 INFO (jetty-launcher-367-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.1.0
[junit4] 2> 584577 INFO (jetty-launcher-367-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 584577 INFO (jetty-launcher-367-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 584577 INFO (jetty-launcher-367-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2019-04-08T05:40:49.940Z
[junit4] 2> 584584 INFO (zkConnectionManagerCallback-371-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 584585 INFO (jetty-launcher-367-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 584592 INFO (jetty-launcher-367-thread-1) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 584592 INFO (jetty-launcher-367-thread-1) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 584593 INFO (jetty-launcher-367-thread-1) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@5137c649, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 584596 INFO (jetty-launcher-367-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 584599 INFO (jetty-launcher-367-thread-2) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 584599 INFO (jetty-launcher-367-thread-2) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 584616 INFO (zkConnectionManagerCallback-373-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 584636 INFO (jetty-launcher-367-thread-2) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@5137c649, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 584637 INFO (jetty-launcher-367-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 584640 INFO (jetty-launcher-367-thread-3) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 584640 INFO (jetty-launcher-367-thread-3) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 584642 INFO (jetty-launcher-367-thread-3) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@5137c649, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 584881 INFO (jetty-launcher-367-thread-2) [ ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
[junit4] 2> 584889 WARN (jetty-launcher-367-thread-2) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@395844b1[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 584892 INFO (jetty-launcher-367-thread-3) [ ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
[junit4] 2> 584913 WARN (jetty-launcher-367-thread-2) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@33bad794[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 584914 INFO (jetty-launcher-367-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46295/solr
[junit4] 2> 584934 WARN (jetty-launcher-367-thread-3) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@2b558d81[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 584948 INFO (zkConnectionManagerCallback-383-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 584949 WARN (jetty-launcher-367-thread-3) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@334e01da[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 584950 INFO (jetty-launcher-367-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46295/solr
[junit4] 2> 584963 INFO (zkConnectionManagerCallback-388-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 584982 INFO (zkConnectionManagerCallback-392-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 584992 INFO (zkConnectionManagerCallback-390-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 585049 INFO (jetty-launcher-367-thread-2) [n:127.0.0.1:35179_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:35179_solr
[junit4] 2> 585050 INFO (jetty-launcher-367-thread-2) [n:127.0.0.1:35179_solr ] o.a.s.c.Overseer Overseer (id=74485011472580616-127.0.0.1:35179_solr-n_0000000000) starting
[junit4] 2> 585053 INFO (jetty-launcher-367-thread-1) [ ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
[junit4] 2> 585068 WARN (jetty-launcher-367-thread-1) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@2f4c890d[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 585088 INFO (zkConnectionManagerCallback-402-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 585096 WARN (jetty-launcher-367-thread-1) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@5453dd7a[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 585097 INFO (jetty-launcher-367-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46295/solr
[junit4] 2> 585098 INFO (jetty-launcher-367-thread-2) [n:127.0.0.1:35179_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:46295/solr ready
[junit4] 2> 585101 INFO (OverseerStateUpdate-74485011472580616-127.0.0.1:35179_solr-n_0000000000) [n:127.0.0.1:35179_solr ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:35179_solr
[junit4] 2> 585101 INFO (jetty-launcher-367-thread-2) [n:127.0.0.1:35179_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35179_solr
[junit4] 2> 585101 INFO (zkConnectionManagerCallback-407-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 585106 INFO (OverseerStateUpdate-74485011472580616-127.0.0.1:35179_solr-n_0000000000) [n:127.0.0.1:35179_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 585157 INFO (jetty-launcher-367-thread-2) [n:127.0.0.1:35179_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 585160 INFO (zkConnectionManagerCallback-409-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 585173 INFO (zkCallback-401-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 585176 INFO (jetty-launcher-367-thread-1) [n:127.0.0.1:40208_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 585179 INFO (jetty-launcher-367-thread-1) [n:127.0.0.1:40208_solr ] o.a.s.c.ZkController Publish node=127.0.0.1:40208_solr as DOWN
[junit4] 2> 585180 INFO (jetty-launcher-367-thread-1) [n:127.0.0.1:40208_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 585180 INFO (jetty-launcher-367-thread-1) [n:127.0.0.1:40208_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40208_solr
[junit4] 2> 585185 INFO (jetty-launcher-367-thread-3) [n:127.0.0.1:38158_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 585210 INFO (jetty-launcher-367-thread-2) [n:127.0.0.1:35179_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5137c649
[junit4] 2> 585213 INFO (jetty-launcher-367-thread-3) [n:127.0.0.1:38158_solr ] o.a.s.c.ZkController Publish node=127.0.0.1:38158_solr as DOWN
[junit4] 2> 585214 INFO (jetty-launcher-367-thread-3) [n:127.0.0.1:38158_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 585214 INFO (jetty-launcher-367-thread-3) [n:127.0.0.1:38158_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38158_solr
[junit4] 2> 585218 INFO (zkCallback-401-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (3)
[junit4] 2> 585220 INFO (jetty-launcher-367-thread-2) [n:127.0.0.1:35179_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5137c649
[junit4] 2> 585220 INFO (jetty-launcher-367-thread-2) [n:127.0.0.1:35179_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5137c649
[junit4] 2> 585221 INFO (jetty-launcher-367-thread-2) [n:127.0.0.1:35179_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest_C6E42D8DDF884DA6-001/tempDir-002/node2/.
[junit4] 2> 585224 INFO (zkCallback-408-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (3)
[junit4] 2> 585230 INFO (zkCallback-389-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (3)
[junit4] 2> 585230 INFO (zkCallback-391-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (3)
[junit4] 2> 585284 INFO (zkConnectionManagerCallback-417-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 585290 INFO (zkConnectionManagerCallback-422-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 585312 INFO (jetty-launcher-367-thread-1) [n:127.0.0.1:40208_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 585312 INFO (jetty-launcher-367-thread-3) [n:127.0.0.1:38158_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 585323 INFO (jetty-launcher-367-thread-1) [n:127.0.0.1:40208_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:46295/solr ready
[junit4] 2> 585323 INFO (jetty-launcher-367-thread-3) [n:127.0.0.1:38158_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:46295/solr ready
[junit4] 2> 585336 INFO (jetty-launcher-367-thread-1) [n:127.0.0.1:40208_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 585352 INFO (jetty-launcher-367-thread-3) [n:127.0.0.1:38158_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 585401 INFO (jetty-launcher-367-thread-1) [n:127.0.0.1:40208_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5137c649
[junit4] 2> 585425 INFO (jetty-launcher-367-thread-1) [n:127.0.0.1:40208_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5137c649
[junit4] 2> 585425 INFO (jetty-launcher-367-thread-1) [n:127.0.0.1:40208_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5137c649
[junit4] 2> 585427 INFO (jetty-launcher-367-thread-1) [n:127.0.0.1:40208_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest_C6E42D8DDF884DA6-001/tempDir-002/node1/.
[junit4] 2> 585429 INFO (jetty-launcher-367-thread-3) [n:127.0.0.1:38158_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5137c649
[junit4] 2> 585440 INFO (jetty-launcher-367-thread-3) [n:127.0.0.1:38158_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5137c649
[junit4] 2> 585440 INFO (jetty-launcher-367-thread-3) [n:127.0.0.1:38158_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5137c649
[junit4] 2> 585442 INFO (jetty-launcher-367-thread-3) [n:127.0.0.1:38158_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest_C6E42D8DDF884DA6-001/tempDir-002/node3/.
[junit4] 2> 585555 INFO (zkConnectionManagerCallback-429-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 585559 INFO (TEST-HdfsAutoAddReplicasIntegrationTest.testSimple-seed#[C6E42D8DDF884DA6]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 585560 INFO (TEST-HdfsAutoAddReplicasIntegrationTest.testSimple-seed#[C6E42D8DDF884DA6]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:46295/solr ready
[junit4] 2> 585590 INFO (qtp523213857-1223) [n:127.0.0.1:40208_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster params={wt=javabin&version=2} status=0 QTime=17
[junit4] 2> 585596 INFO (qtp1539560012-1231) [n:127.0.0.1:35179_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf&maxShardsPerNode=2&autoAddReplicas=true&name=testSimple1&nrtReplicas=2&action=CREATE&numShards=2&createNodeSet=127.0.0.1:35179_solr,127.0.0.1:40208_solr&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 585600 INFO (OverseerThreadFactory-484-thread-1-processing-n:127.0.0.1:35179_solr) [n:127.0.0.1:35179_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection testSimple1
[junit4] 2> 585709 INFO (OverseerStateUpdate-74485011472580616-127.0.0.1:35179_solr-n_0000000000) [n:127.0.0.1:35179_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testSimple1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"testSimple1_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:40208/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 585713 INFO (OverseerStateUpdate-74485011472580616-127.0.0.1:35179_solr-n_0000000000) [n:127.0.0.1:35179_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testSimple1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"testSimple1_shard1_replica_n3",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:35179/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 585716 INFO (OverseerStateUpdate-74485011472580616-127.0.0.1:35179_solr-n_0000000000) [n:127.0.0.1:35179_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testSimple1",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"testSimple1_shard2_replica_n5",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:40208/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 585720 INFO (OverseerStateUpdate-74485011472580616-127.0.0.1:35179_solr-n_0000000000) [n:127.0.0.1:35179_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testSimple1",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"testSimple1_shard2_replica_n7",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:35179/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 585931 INFO (qtp1539560012-1233) [n:127.0.0.1:35179_solr x:testSimple1_shard2_replica_n7] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf&newCollection=true&name=testSimple1_shard2_replica_n7&action=CREATE&numShards=2&collection=testSimple1&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 585931 INFO (qtp1539560012-1233) [n:127.0.0.1:35179_solr x:testSimple1_shard2_replica_n7] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 585942 INFO (qtp523213857-1225) [n:127.0.0.1:40208_solr x:testSimple1_shard2_replica_n5] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node6&collection.configName=conf&newCollection=true&name=testSimple1_shard2_replica_n5&action=CREATE&numShards=2&collection=testSimple1&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 585943 INFO (qtp1539560012-1230) [n:127.0.0.1:35179_solr x:testSimple1_shard1_replica_n3] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=testSimple1_shard1_replica_n3&action=CREATE&numShards=2&collection=testSimple1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 585966 INFO (qtp523213857-1222) [n:127.0.0.1:40208_solr x:testSimple1_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf&newCollection=true&name=testSimple1_shard1_replica_n1&action=CREATE&numShards=2&collection=testSimple1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 586963 INFO (qtp523213857-1225) [n:127.0.0.1:40208_solr c:testSimple1 s:shard2 r:core_node6 x:testSimple1_shard2_replica_n5] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.1.0
[junit4] 2> 586969 INFO (qtp1539560012-1230) [n:127.0.0.1:35179_solr c:testSimple1 s:shard1 r:core_node4 x:testSimple1_shard1_replica_n3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.1.0
[junit4] 2> 586998 INFO (qtp523213857-1225) [n:127.0.0.1:40208_solr c:testSimple1 s:shard2 r:core_node6 x:testSimple1_shard2_replica_n5] o.a.s.s.IndexSchema [testSimple1_shard2_replica_n5] Schema name=minimal
[junit4] 2> 587001 INFO (qtp523213857-1225) [n:127.0.0.1:40208_solr c:testSimple1 s:shard2 r:core_node6 x:testSimple1_shard2_replica_n5] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 587001 INFO (qtp523213857-1225) [n:127.0.0.1:40208_solr c:testSimple1 s:shard2 r:core_node6 x:testSimple1_shard2_replica_n5] o.a.s.c.CoreContainer Creating SolrCore 'testSimple1_shard2_replica_n5' using configuration from collection testSimple1, trusted=true
[junit4] 2> 587005 INFO (qtp1539560012-1233) [n:127.0.0.1:35179_solr c:testSimple1 s:shard2 r:core_node8 x:testSimple1_shard2_replica_n7] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.1.0
[junit4] 2> 587011 INFO (qtp523213857-1225) [n:127.0.0.1:40208_solr c:testSimple1 s:shard2 r:core_node6 x:testSimple1_shard2_replica_n
[...truncated too long message...]
source] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/null704085695
[copy] Copying 240 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/null704085695
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/null704085695
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 59ms :: artifacts dl 4ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 559 minutes 16 seconds
Archiving artifacts
java.lang.InterruptedException: no matches found within 10000
at hudson.FilePath$ValidateAntFileMask.hasMatch(FilePath.java:2847)
at hudson.FilePath$ValidateAntFileMask.invoke(FilePath.java:2726)
at hudson.FilePath$ValidateAntFileMask.invoke(FilePath.java:2707)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3086)
Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to lucene2
at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
at hudson.remoting.Channel.call(Channel.java:955)
at hudson.FilePath.act(FilePath.java:1072)
at hudson.FilePath.act(FilePath.java:1061)
at hudson.FilePath.validateAntFileMask(FilePath.java:2705)
at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
at hudson.model.Run.execute(Run.java:1835)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused: hudson.FilePath$TunneledInterruptedException
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3088)
at hudson.remoting.UserRequest.perform(UserRequest.java:212)
at hudson.remoting.UserRequest.perform(UserRequest.java:54)
at hudson.remoting.Request$2.run(Request.java:369)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Caused: java.lang.InterruptedException: java.lang.InterruptedException: no matches found within 10000
at hudson.FilePath.act(FilePath.java:1074)
at hudson.FilePath.act(FilePath.java:1061)
at hudson.FilePath.validateAntFileMask(FilePath.java:2705)
at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
at hudson.model.Run.execute(Run.java:1835)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
[JENKINS] Lucene-Solr-NightlyTests-8.x - Build # 65 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-8.x/65/
6 tests failed.
FAILED: org.apache.lucene.codecs.lucene80.TestLucene80NormsFormatMergeInstance.testSparseOutliers
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([D38FE8AF9DF17234]:0)
FAILED: junit.framework.TestSuite.org.apache.lucene.codecs.lucene80.TestLucene80NormsFormatMergeInstance
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([D38FE8AF9DF17234]:0)
FAILED: org.apache.solr.cloud.api.collections.ShardSplitTest.test
Error Message:
SPLITSHARD was not successful even after three tries
Stack Trace:
java.lang.AssertionError: SPLITSHARD was not successful even after three tries
at __randomizedtesting.SeedInfo.seed([33053318C9775AF3:BB510CC2678B370B]:0)
at org.junit.Assert.fail(Assert.java:88)
at org.apache.solr.cloud.api.collections.ShardSplitTest.trySplit(ShardSplitTest.java:946)
at org.apache.solr.cloud.api.collections.ShardSplitTest.splitByRouteKeyTest(ShardSplitTest.java:920)
at org.apache.solr.cloud.api.collections.ShardSplitTest.test(ShardSplitTest.java:113)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1082)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:1054)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest.testSimple
Error Message:
Waiting for collection testSimple2 Timeout waiting to see state for collection=testSimple2 :DocCollection(testSimple2//collections/testSimple2/state.json/24)={ "pullReplicas":"0", "replicationFactor":"2", "shards":{ "shard1":{ "range":"80000000-ffffffff", "state":"active", "replicas":{ "core_node3":{ "dataDir":"hdfs://lucene2-us-west.apache.org:38335/solr_hdfs_home/testSimple2/core_node3/data/", "base_url":"https://127.0.0.1:44385/solr", "node_name":"127.0.0.1:44385_solr", "type":"NRT", "force_set_state":"false", "ulogDir":"hdfs://lucene2-us-west.apache.org:38335/solr_hdfs_home/testSimple2/core_node3/data/tlog", "core":"testSimple2_shard1_replica_n1", "shared_storage":"true", "state":"active", "leader":"true"}, "core_node5":{ "dataDir":"hdfs://lucene2-us-west.apache.org:38335/solr_hdfs_home/testSimple2/core_node5/data/", "base_url":"https://127.0.0.1:46506/solr", "node_name":"127.0.0.1:46506_solr", "type":"NRT", "force_set_state":"false", "ulogDir":"hdfs://lucene2-us-west.apache.org:38335/solr_hdfs_home/testSimple2/core_node5/data/tlog", "core":"testSimple2_shard1_replica_n2", "shared_storage":"true", "state":"down"}}}, "shard2":{ "range":"0-7fffffff", "state":"active", "replicas":{ "core_node7":{ "dataDir":"hdfs://lucene2-us-west.apache.org:38335/solr_hdfs_home/testSimple2/core_node7/data/", "base_url":"https://127.0.0.1:44385/solr", "node_name":"127.0.0.1:44385_solr", "type":"NRT", "force_set_state":"false", "ulogDir":"hdfs://lucene2-us-west.apache.org:38335/solr_hdfs_home/testSimple2/core_node7/data/tlog", "core":"testSimple2_shard2_replica_n4", "shared_storage":"true", "state":"active", "leader":"true"}, "core_node8":{ "dataDir":"hdfs://lucene2-us-west.apache.org:38335/solr_hdfs_home/testSimple2/core_node8/data/", "base_url":"https://127.0.0.1:46506/solr", "node_name":"127.0.0.1:46506_solr", "type":"NRT", "force_set_state":"false", "ulogDir":"hdfs://lucene2-us-west.apache.org:38335/solr_hdfs_home/testSimple2/core_node8/data/tlog", "core":"testSimple2_shard2_replica_n6", "shared_storage":"true", "state":"down"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"2", "autoAddReplicas":"true", "nrtReplicas":"2", "tlogReplicas":"0"} Live Nodes: [127.0.0.1:40408_solr, 127.0.0.1:44385_solr] Last available state: DocCollection(testSimple2//collections/testSimple2/state.json/24)={ "pullReplicas":"0", "replicationFactor":"2", "shards":{ "shard1":{ "range":"80000000-ffffffff", "state":"active", "replicas":{ "core_node3":{ "dataDir":"hdfs://lucene2-us-west.apache.org:38335/solr_hdfs_home/testSimple2/core_node3/data/", "base_url":"https://127.0.0.1:44385/solr", "node_name":"127.0.0.1:44385_solr", "type":"NRT", "force_set_state":"false", "ulogDir":"hdfs://lucene2-us-west.apache.org:38335/solr_hdfs_home/testSimple2/core_node3/data/tlog", "core":"testSimple2_shard1_replica_n1", "shared_storage":"true", "state":"active", "leader":"true"}, "core_node5":{ "dataDir":"hdfs://lucene2-us-west.apache.org:38335/solr_hdfs_home/testSimple2/core_node5/data/", "base_url":"https://127.0.0.1:46506/solr", "node_name":"127.0.0.1:46506_solr", "type":"NRT", "force_set_state":"false", "ulogDir":"hdfs://lucene2-us-west.apache.org:38335/solr_hdfs_home/testSimple2/core_node5/data/tlog", "core":"testSimple2_shard1_replica_n2", "shared_storage":"true", "state":"down"}}}, "shard2":{ "range":"0-7fffffff", "state":"active", "replicas":{ "core_node7":{ "dataDir":"hdfs://lucene2-us-west.apache.org:38335/solr_hdfs_home/testSimple2/core_node7/data/", "base_url":"https://127.0.0.1:44385/solr", "node_name":"127.0.0.1:44385_solr", "type":"NRT", "force_set_state":"false", "ulogDir":"hdfs://lucene2-us-west.apache.org:38335/solr_hdfs_home/testSimple2/core_node7/data/tlog", "core":"testSimple2_shard2_replica_n4", "shared_storage":"true", "state":"active", "leader":"true"}, "core_node8":{ "dataDir":"hdfs://lucene2-us-west.apache.org:38335/solr_hdfs_home/testSimple2/core_node8/data/", "base_url":"https://127.0.0.1:46506/solr", "node_name":"127.0.0.1:46506_solr", "type":"NRT", "force_set_state":"false", "ulogDir":"hdfs://lucene2-us-west.apache.org:38335/solr_hdfs_home/testSimple2/core_node8/data/tlog", "core":"testSimple2_shard2_replica_n6", "shared_storage":"true", "state":"down"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"2", "autoAddReplicas":"true", "nrtReplicas":"2", "tlogReplicas":"0"}
Stack Trace:
java.lang.AssertionError: Waiting for collection testSimple2
Timeout waiting to see state for collection=testSimple2 :DocCollection(testSimple2//collections/testSimple2/state.json/24)={
"pullReplicas":"0",
"replicationFactor":"2",
"shards":{
"shard1":{
"range":"80000000-ffffffff",
"state":"active",
"replicas":{
"core_node3":{
"dataDir":"hdfs://lucene2-us-west.apache.org:38335/solr_hdfs_home/testSimple2/core_node3/data/",
"base_url":"https://127.0.0.1:44385/solr",
"node_name":"127.0.0.1:44385_solr",
"type":"NRT",
"force_set_state":"false",
"ulogDir":"hdfs://lucene2-us-west.apache.org:38335/solr_hdfs_home/testSimple2/core_node3/data/tlog",
"core":"testSimple2_shard1_replica_n1",
"shared_storage":"true",
"state":"active",
"leader":"true"},
"core_node5":{
"dataDir":"hdfs://lucene2-us-west.apache.org:38335/solr_hdfs_home/testSimple2/core_node5/data/",
"base_url":"https://127.0.0.1:46506/solr",
"node_name":"127.0.0.1:46506_solr",
"type":"NRT",
"force_set_state":"false",
"ulogDir":"hdfs://lucene2-us-west.apache.org:38335/solr_hdfs_home/testSimple2/core_node5/data/tlog",
"core":"testSimple2_shard1_replica_n2",
"shared_storage":"true",
"state":"down"}}},
"shard2":{
"range":"0-7fffffff",
"state":"active",
"replicas":{
"core_node7":{
"dataDir":"hdfs://lucene2-us-west.apache.org:38335/solr_hdfs_home/testSimple2/core_node7/data/",
"base_url":"https://127.0.0.1:44385/solr",
"node_name":"127.0.0.1:44385_solr",
"type":"NRT",
"force_set_state":"false",
"ulogDir":"hdfs://lucene2-us-west.apache.org:38335/solr_hdfs_home/testSimple2/core_node7/data/tlog",
"core":"testSimple2_shard2_replica_n4",
"shared_storage":"true",
"state":"active",
"leader":"true"},
"core_node8":{
"dataDir":"hdfs://lucene2-us-west.apache.org:38335/solr_hdfs_home/testSimple2/core_node8/data/",
"base_url":"https://127.0.0.1:46506/solr",
"node_name":"127.0.0.1:46506_solr",
"type":"NRT",
"force_set_state":"false",
"ulogDir":"hdfs://lucene2-us-west.apache.org:38335/solr_hdfs_home/testSimple2/core_node8/data/tlog",
"core":"testSimple2_shard2_replica_n6",
"shared_storage":"true",
"state":"down"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"2",
"autoAddReplicas":"true",
"nrtReplicas":"2",
"tlogReplicas":"0"}
Live Nodes: [127.0.0.1:40408_solr, 127.0.0.1:44385_solr]
Last available state: DocCollection(testSimple2//collections/testSimple2/state.json/24)={
"pullReplicas":"0",
"replicationFactor":"2",
"shards":{
"shard1":{
"range":"80000000-ffffffff",
"state":"active",
"replicas":{
"core_node3":{
"dataDir":"hdfs://lucene2-us-west.apache.org:38335/solr_hdfs_home/testSimple2/core_node3/data/",
"base_url":"https://127.0.0.1:44385/solr",
"node_name":"127.0.0.1:44385_solr",
"type":"NRT",
"force_set_state":"false",
"ulogDir":"hdfs://lucene2-us-west.apache.org:38335/solr_hdfs_home/testSimple2/core_node3/data/tlog",
"core":"testSimple2_shard1_replica_n1",
"shared_storage":"true",
"state":"active",
"leader":"true"},
"core_node5":{
"dataDir":"hdfs://lucene2-us-west.apache.org:38335/solr_hdfs_home/testSimple2/core_node5/data/",
"base_url":"https://127.0.0.1:46506/solr",
"node_name":"127.0.0.1:46506_solr",
"type":"NRT",
"force_set_state":"false",
"ulogDir":"hdfs://lucene2-us-west.apache.org:38335/solr_hdfs_home/testSimple2/core_node5/data/tlog",
"core":"testSimple2_shard1_replica_n2",
"shared_storage":"true",
"state":"down"}}},
"shard2":{
"range":"0-7fffffff",
"state":"active",
"replicas":{
"core_node7":{
"dataDir":"hdfs://lucene2-us-west.apache.org:38335/solr_hdfs_home/testSimple2/core_node7/data/",
"base_url":"https://127.0.0.1:44385/solr",
"node_name":"127.0.0.1:44385_solr",
"type":"NRT",
"force_set_state":"false",
"ulogDir":"hdfs://lucene2-us-west.apache.org:38335/solr_hdfs_home/testSimple2/core_node7/data/tlog",
"core":"testSimple2_shard2_replica_n4",
"shared_storage":"true",
"state":"active",
"leader":"true"},
"core_node8":{
"dataDir":"hdfs://lucene2-us-west.apache.org:38335/solr_hdfs_home/testSimple2/core_node8/data/",
"base_url":"https://127.0.0.1:46506/solr",
"node_name":"127.0.0.1:46506_solr",
"type":"NRT",
"force_set_state":"false",
"ulogDir":"hdfs://lucene2-us-west.apache.org:38335/solr_hdfs_home/testSimple2/core_node8/data/tlog",
"core":"testSimple2_shard2_replica_n6",
"shared_storage":"true",
"state":"down"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"2",
"autoAddReplicas":"true",
"nrtReplicas":"2",
"tlogReplicas":"0"}
at __randomizedtesting.SeedInfo.seed([33053318C9775AF3:BB617E6EE848E22]:0)
at org.junit.Assert.fail(Assert.java:88)
at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:310)
at org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple(AutoAddReplicasIntegrationTest.java:169)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest
Error Message:
ObjectTracker found 4 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory, InternalHttpClient, SolrCore] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348) at org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:517) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:968) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:883) at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1245) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1155) at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92) at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360) at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:396) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:180) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199) at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:784) at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:750) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:510) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:397) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:343) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:165) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:703) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:502) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:411) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:305) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:159) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348) at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:99) at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:779) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:976) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:883) at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1245) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1155) at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92) at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360) at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:396) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:180) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199) at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:784) at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:750) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:510) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:397) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:343) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:165) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:703) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:502) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:411) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:305) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:159) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:321) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:330) at org.apache.solr.handler.IndexFetcher.createHttpClient(IndexFetcher.java:230) at org.apache.solr.handler.IndexFetcher.<init>(IndexFetcher.java:272) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:420) at org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:237) at org.apache.solr.cloud.RecoveryStrategy.doSyncOrReplicateRecovery(RecoveryStrategy.java:652) at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:326) at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:307) at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.core.SolrCore at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1063) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:883) at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1245) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1155) at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92) at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360) at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:396) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:180) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199) at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:784) at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:750) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:510) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:397) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:343) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:165) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:703) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:502) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:411) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:305) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:159) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683) at java.lang.Thread.run(Thread.java:748) expected null, but was:<ObjectTracker found 4 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory, InternalHttpClient, SolrCore] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348) at org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:517) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:968) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:883) at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1245) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1155) at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92) at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360) at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:396) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:180) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199) at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:784) at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:750) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:510) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:397) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:343) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:165) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:703) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:502) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:411) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:305) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:159) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348) at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:99) at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:779) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:976) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:883) at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1245) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1155) at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92) at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360) at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:396) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:180) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199) at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:784) at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:750) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:510) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:397) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:343) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:165) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:703) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:502) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:411) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:305) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:159) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:321) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:330) at org.apache.solr.handler.IndexFetcher.createHttpClient(IndexFetcher.java:230) at org.apache.solr.handler.IndexFetcher.<init>(IndexFetcher.java:272) at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:420) at org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:237) at org.apache.solr.cloud.RecoveryStrategy.doSyncOrReplicateRecovery(RecoveryStrategy.java:652) at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:326) at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:307) at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.core.SolrCore at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1063) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:883) at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1245) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1155) at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92) at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360) at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:396) at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:180) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199) at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:784) at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:750) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:510) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:397) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:343) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:165) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:703) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:502) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:411) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:305) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:159) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683) at java.lang.Thread.run(Thread.java:748) >
Stack Trace:
java.lang.AssertionError: ObjectTracker found 4 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory, InternalHttpClient, SolrCore]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
at org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:517)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:968)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:883)
at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1245)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1155)
at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92)
at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360)
at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:396)
at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:180)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:784)
at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:750)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:510)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:397)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:343)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:165)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:703)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:502)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:411)
at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:305)
at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:159)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
at java.lang.Thread.run(Thread.java:748)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:99)
at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:779)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:976)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:883)
at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1245)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1155)
at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92)
at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360)
at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:396)
at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:180)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:784)
at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:750)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:510)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:397)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:343)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:165)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:703)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:502)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:411)
at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:305)
at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:159)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
at java.lang.Thread.run(Thread.java:748)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:321)
at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:330)
at org.apache.solr.handler.IndexFetcher.createHttpClient(IndexFetcher.java:230)
at org.apache.solr.handler.IndexFetcher.<init>(IndexFetcher.java:272)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:420)
at org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:237)
at org.apache.solr.cloud.RecoveryStrategy.doSyncOrReplicateRecovery(RecoveryStrategy.java:652)
at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:326)
at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:307)
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.core.SolrCore
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1063)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:883)
at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1245)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1155)
at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92)
at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360)
at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:396)
at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:180)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:784)
at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:750)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:510)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:397)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:343)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:165)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:703)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:502)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:411)
at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:305)
at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:159)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
at java.lang.Thread.run(Thread.java:748)
expected null, but was:<ObjectTracker found 4 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory, InternalHttpClient, SolrCore]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
at org.apache.solr.core.SolrCore.initSnapshotMetaDataManager(SolrCore.java:517)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:968)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:883)
at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1245)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1155)
at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92)
at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360)
at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:396)
at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:180)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:784)
at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:750)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:510)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:397)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:343)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:165)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:703)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:502)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:411)
at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:305)
at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:159)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
at java.lang.Thread.run(Thread.java:748)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.NRTCachingDirectory
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:99)
at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:779)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:976)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:883)
at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1245)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1155)
at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92)
at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360)
at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:396)
at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:180)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:784)
at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:750)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:510)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:397)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:343)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:165)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:703)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:502)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:411)
at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:305)
at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:159)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
at java.lang.Thread.run(Thread.java:748)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:321)
at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:330)
at org.apache.solr.handler.IndexFetcher.createHttpClient(IndexFetcher.java:230)
at org.apache.solr.handler.IndexFetcher.<init>(IndexFetcher.java:272)
at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:420)
at org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:237)
at org.apache.solr.cloud.RecoveryStrategy.doSyncOrReplicateRecovery(RecoveryStrategy.java:652)
at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:326)
at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:307)
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.core.SolrCore
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1063)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:883)
at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1245)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1155)
at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92)
at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360)
at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:396)
at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:180)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:784)
at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:750)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:510)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:397)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:343)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:165)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:703)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:502)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:411)
at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:305)
at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:159)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
at java.lang.Thread.run(Thread.java:748)
>
at __randomizedtesting.SeedInfo.seed([33053318C9775AF3]:0)
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotNull(Assert.java:755)
at org.junit.Assert.assertNull(Assert.java:737)
at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:325)
at sun.reflect.GeneratedMethodAccessor108.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:901)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.client.solrj.request.TestV2Request.testCloudSolrClient
Error Message:
Error from server at http://127.0.0.1:45674/solr: 4 out of 5 the property overlay to be of version 0 within 30 seconds! Failed cores: [http://127.0.0.1:44639/solr/test_shard2_replica_n3/, http://127.0.0.1:46320/solr/test_shard2_replica_n4/, http://127.0.0.1:37559/solr/test_shard1_replica_n1/, http://127.0.0.1:45674/solr/test_shard1_replica_n2/]
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteExecutionException: Error from server at http://127.0.0.1:45674/solr: 4 out of 5 the property overlay to be of version 0 within 30 seconds! Failed cores: [http://127.0.0.1:44639/solr/test_shard2_replica_n3/, http://127.0.0.1:46320/solr/test_shard2_replica_n4/, http://127.0.0.1:37559/solr/test_shard1_replica_n1/, http://127.0.0.1:45674/solr/test_shard1_replica_n2/]
at __randomizedtesting.SeedInfo.seed([F9607E63765DD1E1:6096F5D3D3EE598A]:0)
at org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteExecutionException.create(BaseHttpSolrClient.java:65)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:626)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBSolrClient.doRequest(LBSolrClient.java:368)
at org.apache.solr.client.solrj.impl.LBSolrClient.request(LBSolrClient.java:296)
at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.sendRequest(BaseCloudSolrClient.java:1055)
at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.requestWithRetryOnStaleState(BaseCloudSolrClient.java:830)
at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.request(BaseCloudSolrClient.java:763)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1274)
at org.apache.solr.client.solrj.request.TestV2Request.assertSuccess(TestV2Request.java:55)
at org.apache.solr.client.solrj.request.TestV2Request.doTest(TestV2Request.java:102)
at org.apache.solr.client.solrj.request.TestV2Request.testCloudSolrClient(TestV2Request.java:83)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 1695 lines...]
[junit4] Suite: org.apache.lucene.codecs.lucene80.TestLucene80NormsFormatMergeInstance
[junit4] IGNOR/A 0.05s J1 | TestLucene80NormsFormatMergeInstance.testMergeStability
[junit4] > Assumption #1: The MockRandom PF randomizes content on the fly, so we can't check it
[junit4] 2> Apr 07, 2019 7:18:57 AM com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
[junit4] 2> WARNING: Suite execution timed out: org.apache.lucene.codecs.lucene80.TestLucene80NormsFormatMergeInstance
[junit4] 2> 1) Thread[id=9, name=JUnit4-serializer-daemon, state=TIMED_WAITING, group=main]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:50)
[junit4] 2> 2) Thread[id=1067, name=SUITE-TestLucene80NormsFormatMergeInstance-seed#[D38FE8AF9DF17234], state=RUNNABLE, group=TGRP-TestLucene80NormsFormatMergeInstance]
[junit4] 2> at java.lang.Thread.getStackTrace(Thread.java:1559)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:696)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:693)
[junit4] 2> at java.security.AccessController.doPrivileged(Native Method)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getStackTrace(ThreadLeakControl.java:693)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.getThreadsWithTraces(ThreadLeakControl.java:709)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:689)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:65)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:415)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:708)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:138)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:629)
[junit4] 2> 3) Thread[id=1, name=main, state=WAITING, group=main]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Thread.join(Thread.java:1252)
[junit4] 2> at java.lang.Thread.join(Thread.java:1326)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:639)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:496)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:269)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:394)
[junit4] 2> at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:13)
[junit4] 2> 4) Thread[id=1068, name=TEST-TestLucene80NormsFormatMergeInstance.testSparseOutliers-seed#[D38FE8AF9DF17234], state=TIMED_WAITING, group=TGRP-TestLucene80NormsFormatMergeInstance]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at org.apache.lucene.index.IndexWriter.doWait(IndexWriter.java:4685)
[junit4] 2> at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:2011)
[junit4] 2> at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1939)
[junit4] 2> at org.apache.lucene.index.RandomIndexWriter.forceMerge(RandomIndexWriter.java:500)
[junit4] 2> at org.apache.lucene.index.BaseNormsFormatTestCase.doTestNormsVersusDocValues(BaseNormsFormatTestCase.java:499)
[junit4] 2> at org.apache.lucene.index.BaseNormsFormatTestCase.testSparseOutliers(BaseNormsFormatTestCase.java:290)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:498)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant te
[...truncated too long message...]
rce] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/null1744671832
[copy] Copying 240 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/null1744671832
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/null1744671832
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 35ms :: artifacts dl 5ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 603 minutes 57 seconds
Archiving artifacts
java.lang.InterruptedException: no matches found within 10000
at hudson.FilePath$ValidateAntFileMask.hasMatch(FilePath.java:2847)
at hudson.FilePath$ValidateAntFileMask.invoke(FilePath.java:2726)
at hudson.FilePath$ValidateAntFileMask.invoke(FilePath.java:2707)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3086)
Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to lucene2
at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
at hudson.remoting.Channel.call(Channel.java:955)
at hudson.FilePath.act(FilePath.java:1072)
at hudson.FilePath.act(FilePath.java:1061)
at hudson.FilePath.validateAntFileMask(FilePath.java:2705)
at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
at hudson.model.Run.execute(Run.java:1835)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused: hudson.FilePath$TunneledInterruptedException
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3088)
at hudson.remoting.UserRequest.perform(UserRequest.java:212)
at hudson.remoting.UserRequest.perform(UserRequest.java:54)
at hudson.remoting.Request$2.run(Request.java:369)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Caused: java.lang.InterruptedException: java.lang.InterruptedException: no matches found within 10000
at hudson.FilePath.act(FilePath.java:1074)
at hudson.FilePath.act(FilePath.java:1061)
at hudson.FilePath.validateAntFileMask(FilePath.java:2705)
at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
at hudson.model.Run.execute(Run.java:1835)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
[JENKINS] Lucene-Solr-NightlyTests-8.x - Build # 64 - Still
unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-8.x/64/
4 tests failed.
FAILED: org.apache.solr.cloud.hdfs.HdfsTlogReplayBufferedWhileIndexingTest.test
Error Message:
There are still nodes recoverying - waited for 600 seconds
Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 600 seconds
at __randomizedtesting.SeedInfo.seed([E759039FB489C8A5:6F0D3C451A75A55D]:0)
at org.junit.Assert.fail(Assert.java:88)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:195)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:1038)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForThingsToLevelOut(AbstractFullDistribZkTestBase.java:1595)
at org.apache.solr.cloud.TlogReplayBufferedWhileIndexingTest.test(TlogReplayBufferedWhileIndexingTest.java:97)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1082)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:1054)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigReplication
Error Message:
expected:<1554553608390> but was:<0>
Stack Trace:
java.lang.AssertionError: expected:<1554553608390> but was:<0>
at __randomizedtesting.SeedInfo.seed([E759039FB489C8A5:F31158CA978E75BB]:0)
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:834)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:144)
at org.apache.solr.handler.TestReplicationHandler.assertVersions(TestReplicationHandler.java:1091)
at org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigReplication(TestReplicationHandler.java:510)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest: 1) Thread[id=165226, name=zkConnectionManagerCallback-77027-thread-1, state=WAITING, group=TGRP-AutoscalingHistoryHandlerTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest:
1) Thread[id=165226, name=zkConnectionManagerCallback-77027-thread-1, state=WAITING, group=TGRP-AutoscalingHistoryHandlerTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([E759039FB489C8A5]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=165226, name=zkConnectionManagerCallback-77027-thread-1, state=WAITING, group=TGRP-AutoscalingHistoryHandlerTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=165226, name=zkConnectionManagerCallback-77027-thread-1, state=WAITING, group=TGRP-AutoscalingHistoryHandlerTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([E759039FB489C8A5]:0)
Build Log:
[...truncated 14055 lines...]
[junit4] Suite: org.apache.solr.handler.TestReplicationHandler
[junit4] 2> 1324299 INFO (SUITE-TestReplicationHandler-seed#[E759039FB489C8A5]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_E759039FB489C8A5-001/init-core-data-001
[junit4] 2> 1324300 WARN (SUITE-TestReplicationHandler-seed#[E759039FB489C8A5]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=93 numCloses=93
[junit4] 2> 1324324 INFO (SUITE-TestReplicationHandler-seed#[E759039FB489C8A5]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 1324326 INFO (SUITE-TestReplicationHandler-seed#[E759039FB489C8A5]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=None)
[junit4] 2> 1324336 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testFetchIndexShouldReportErrorsWhenTheyOccur
[junit4] 2> 1324337 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_E759039FB489C8A5-001/solr-instance-001/collection1
[junit4] 2> 1324340 WARN (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.e.j.s.AbstractConnector Ignoring deprecated socket close linger time
[junit4] 2> 1324374 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.c.s.e.JettySolrRunner Start Jetty (original configured port=0)
[junit4] 2> 1324374 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
[junit4] 2> 1324374 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.e.j.s.Server jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: c4550056e785fb5665914545889f21dc136ad9e6; jvm 1.8.0_191-b12
[junit4] 2> 1324541 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1324541 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1324541 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 1324541 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5cd0d937{/solr,null,AVAILABLE}
[junit4] 2> 1324543 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@4574f9d7{HTTP/1.1,[http/1.1, h2c]}{127.0.0.1:42544}
[junit4] 2> 1324543 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.e.j.s.Server Started @1324620ms
[junit4] 2> 1324543 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_E759039FB489C8A5-001/solr-instance-001/collection1/data, hostContext=/solr, hostPort=42544}
[junit4] 2> 1324556 ERROR (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1324556 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 1324556 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.1.0
[junit4] 2> 1324556 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 1324556 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1324557 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2019-04-06T12:23:18.028Z
[junit4] 2> 1324558 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_E759039FB489C8A5-001/solr-instance-001/solr.xml
[junit4] 2> 1324666 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 1324666 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 1324667 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@6d3255be, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 1325359 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
[junit4] 2> 1325361 WARN (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@1b805ad1[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1325411 WARN (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@296d3300[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1325552 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 1325552 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 1325626 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6d3255be
[junit4] 2> 1325712 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6d3255be
[junit4] 2> 1325712 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6d3255be
[junit4] 2> 1325727 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_E759039FB489C8A5-001/solr-instance-001/.
[junit4] 2> 1325727 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1325909 INFO (coreLoadExecutor-1184-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.1.0
[junit4] 2> 1325946 INFO (coreLoadExecutor-1184-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1326028 INFO (coreLoadExecutor-1184-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
[junit4] 2> 1326028 INFO (coreLoadExecutor-1184-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_E759039FB489C8A5-001/solr-instance-001/./collection1, trusted=true
[junit4] 2> 1326028 INFO (coreLoadExecutor-1184-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6d3255be
[junit4] 2> 1326029 INFO (coreLoadExecutor-1184-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_E759039FB489C8A5-001/solr-instance-001/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_E759039FB489C8A5-001/solr-instance-001/./collection1/data/]
[junit4] 2> 1326030 INFO (coreLoadExecutor-1184-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=17, maxMergeAtOnceExplicit=46, maxMergedSegmentMB=1.6875, floorSegmentMB=1.9091796875, forceMergeDeletesPctAllowed=10.911964320562646, segmentsPerTier=7.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6580946644050136, deletesPctAllowed=27.511140433728293
[junit4] 2> 1326312 INFO (coreLoadExecutor-1184-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1326312 INFO (coreLoadExecutor-1184-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1326313 INFO (coreLoadExecutor-1184-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=34, maxMergeAtOnceExplicit=46, maxMergedSegmentMB=53.6416015625, floorSegmentMB=1.419921875, forceMergeDeletesPctAllowed=24.052583869079925, segmentsPerTier=38.0, maxCFSSegmentSizeMB=1.8232421875, noCFSRatio=0.0, deletesPctAllowed=30.639370724224737
[junit4] 2> 1326313 INFO (coreLoadExecutor-1184-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@7f3228a7[collection1] main]
[junit4] 2> 1326313 INFO (coreLoadExecutor-1184-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_E759039FB489C8A5-001/solr-instance-001/collection1/conf
[junit4] 2> 1326330 INFO (coreLoadExecutor-1184-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema.xml,xslt/dummy.xsl
[junit4] 2> 1326330 INFO (coreLoadExecutor-1184-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 1326369 INFO (searcherExecutor-1185-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@7f3228a7[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1326387 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_E759039FB489C8A5-001/solr-instance-002/collection1
[junit4] 2> 1326390 WARN (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.e.j.s.AbstractConnector Ignoring deprecated socket close linger time
[junit4] 2> 1326390 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.c.s.e.JettySolrRunner Start Jetty (original configured port=0)
[junit4] 2> 1326390 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
[junit4] 2> 1326390 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.e.j.s.Server jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: c4550056e785fb5665914545889f21dc136ad9e6; jvm 1.8.0_191-b12
[junit4] 2> 1326487 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1326487 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1326487 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 1326487 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4dc487e0{/solr,null,AVAILABLE}
[junit4] 2> 1326504 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@9d5c350{HTTP/1.1,[http/1.1, h2c]}{127.0.0.1:33929}
[junit4] 2> 1326504 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.e.j.s.Server Started @1326582ms
[junit4] 2> 1326504 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_E759039FB489C8A5-001/solr-instance-002/collection1/data, hostContext=/solr, hostPort=33929}
[junit4] 2> 1326505 ERROR (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1326505 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 1326505 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.1.0
[junit4] 2> 1326505 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 1326505 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1326505 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2019-04-06T12:23:19.976Z
[junit4] 2> 1326505 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_E759039FB489C8A5-001/solr-instance-002/solr.xml
[junit4] 2> 1326509 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 1326509 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 1326510 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@6d3255be, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 1326984 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
[junit4] 2> 1327020 WARN (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@35f255b4[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1327023 WARN (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@dab74a5[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1327099 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 1327099 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 1327181 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6d3255be
[junit4] 2> 1327283 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6d3255be
[junit4] 2> 1327283 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6d3255be
[junit4] 2> 1327285 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_E759039FB489C8A5-001/solr-instance-002/.
[junit4] 2> 1327285 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1327466 INFO (coreLoadExecutor-1197-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.1.0
[junit4] 2> 1327520 INFO (coreLoadExecutor-1197-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1327652 INFO (coreLoadExecutor-1197-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
[junit4] 2> 1327652 INFO (coreLoadExecutor-1197-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_E759039FB489C8A5-001/solr-instance-002/./collection1, trusted=true
[junit4] 2> 1327653 INFO (coreLoadExecutor-1197-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6d3255be
[junit4] 2> 1327653 INFO (coreLoadExecutor-1197-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_E759039FB489C8A5-001/solr-instance-002/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_E759039FB489C8A5-001/solr-instance-002/./collection1/data/]
[junit4] 2> 1327654 INFO (coreLoadExecutor-1197-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=17, maxMergeAtOnceExplicit=46, maxMergedSegmentMB=1.6875, floorSegmentMB=1.9091796875, forceMergeDeletesPctAllowed=10.911964320562646, segmentsPerTier=7.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6580946644050136, deletesPctAllowed=27.511140433728293
[junit4] 2> 1327859 INFO (coreLoadExecutor-1197-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1327859 INFO (coreLoadExecutor-1197-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1327860 INFO (coreLoadExecutor-1197-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=34, maxMergeAtOnceExplicit=46, maxMergedSegmentMB=53.6416015625, floorSegmentMB=1.419921875, forceMergeDeletesPctAllowed=24.052583869079925, segmentsPerTier=38.0, maxCFSSegmentSizeMB=1.8232421875, noCFSRatio=0.0, deletesPctAllowed=30.639370724224737
[junit4] 2> 1327860 INFO (coreLoadExecutor-1197-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2dce290f[collection1] main]
[junit4] 2> 1327860 INFO (coreLoadExecutor-1197-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_E759039FB489C8A5-001/solr-instance-002/collection1/conf
[junit4] 2> 1327886 INFO (coreLoadExecutor-1197-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 1000ms
[junit4] 2> 1327886 INFO (coreLoadExecutor-1197-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 1327903 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1599810150
[junit4] 2> 1327904 INFO (coreCloseExecutor-1203-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@41b0757d
[junit4] 2> 1327904 INFO (coreCloseExecutor-1203-thread-1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=41b0757d
[junit4] 2> 1327904 INFO (coreCloseExecutor-1203-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@731e153e: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@edc3d64
[junit4] 2> 1327984 INFO (searcherExecutor-1198-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2dce290f[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1328145 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 1328145 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@be5ecbb: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@5b17095e
[junit4] 2> 1328148 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 1328148 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@47a92f0d: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@c8facb9
[junit4] 2> 1328203 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 1328203 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@118ee73: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@44d56aca
[junit4] 2> 1328384 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@4574f9d7{HTTP/1.1,[http/1.1, h2c]}{127.0.0.1:0}
[junit4] 2> 1328384 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5cd0d937{/solr,null,UNAVAILABLE}
[junit4] 2> 1328384 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 1328434 WARN (indexFetcher-1202-thread-1) [ ] o.a.s.h.IndexFetcher Master at: http://127.0.0.1:42544/solr/collection1 is not available. Index fetch failed by exception: org.apache.solr.common.SolrException: Server refused connection at: http://127.0.0.1:42544/solr/collection1
[junit4] 2> 1328564 WARN (explicit-fetchindex-cmd) [ ] o.a.s.h.IndexFetcher Master at: http://127.0.0.1:42544/solr is not available. Index fetch failed by exception: org.apache.solr.common.SolrException: Server refused connection at: http://127.0.0.1:42544/solr
[junit4] 2> 1328564 INFO (qtp22878226-3476) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={wait=true&qt=/replication&masterUrl=http://127.0.0.1:42544/solr&wt=javabin&version=2&command=fetchindex} status=0 QTime=84
[junit4] 2> 1328565 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testFetchIndexShouldReportErrorsWhenTheyOccur
[junit4] 2> 1328566 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1060662318
[junit4] 2> 1328635 INFO (coreCloseExecutor-1205-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@605bc062
[junit4] 2> 1328635 INFO (coreCloseExecutor-1205-thread-1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=605bc062
[junit4] 2> 1328636 INFO (coreCloseExecutor-1205-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2eebe78f: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@76fd5be6
[junit4] 2> 1328704 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 1328704 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4b3ed4b9: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@127c0bde
[junit4] 2> 1328708 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 1328708 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@684b5d96: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@c8facb9
[junit4] 2> 1328710 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 1328710 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4d449218: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@44d56aca
[junit4] 2> 1328767 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@9d5c350{HTTP/1.1,[http/1.1, h2c]}{127.0.0.1:0}
[junit4] 2> 1328767 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@4dc487e0{/solr,null,UNAVAILABLE}
[junit4] 2> 1328767 INFO (TEST-TestReplicationHandler.testFetchIndexShouldReportErrorsWhenTheyOccur-seed#[E759039FB489C8A5]) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 1328781 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.SolrTestCaseJ4 ###Starting doTestReplicateAfterStartup
[junit4] 2> 1328794 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_E759039FB489C8A5-001/solr-instance-003/collection1
[junit4] 2> 1328796 WARN (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.e.j.s.AbstractConnector Ignoring deprecated socket close linger time
[junit4] 2> 1328796 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.c.s.e.JettySolrRunner Start Jetty (original configured port=0)
[junit4] 2> 1328796 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
[junit4] 2> 1328796 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.e.j.s.Server jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: c4550056e785fb5665914545889f21dc136ad9e6; jvm 1.8.0_191-b12
[junit4] 2> 1328830 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1328830 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1328830 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 1328831 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@43019b71{/solr,null,AVAILABLE}
[junit4] 2> 1328850 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@4d51f573{HTTP/1.1,[http/1.1, h2c]}{127.0.0.1:42560}
[junit4] 2> 1328850 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.e.j.s.Server Started @1328927ms
[junit4] 2> 1328850 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_E759039FB489C8A5-001/solr-instance-003/collection1/data, hostContext=/solr, hostPort=42560}
[junit4] 2> 1328851 ERROR (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1328851 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 1328851 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.1.0
[junit4] 2> 1328851 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 1328851 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1328851 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2019-04-06T12:23:22.322Z
[junit4] 2> 1328851 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_E759039FB489C8A5-001/solr-instance-003/solr.xml
[junit4] 2> 1328867 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 1328867 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 1328868 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@6d3255be, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 1329435 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
[junit4] 2> 1329437 WARN (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@2a279f67[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1329525 WARN (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@4a1af758[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1329608 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 1329608 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 1329668 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6d3255be
[junit4] 2> 1329700 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6d3255be
[junit4] 2> 1329700 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6d3255be
[junit4] 2> 1329718 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_E759039FB489C8A5-001/solr-instance-003/.
[junit4] 2> 1329718 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1329984 INFO (coreLoadExecutor-1215-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.1.0
[junit4] 2> 1330214 INFO (coreLoadExecutor-1215-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1330273 INFO (coreLoadExecutor-1215-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
[junit4] 2> 1330273 INFO (coreLoadExecutor-1215-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_E759039FB489C8A5-001/solr-instance-003/./collection1, trusted=true
[junit4] 2> 1330274 INFO (coreLoadExecutor-1215-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6d3255be
[junit4] 2> 1330274 INFO (coreLoadExecutor-1215-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_E759039FB489C8A5-001/solr-instance-003/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_E759039FB489C8A5-001/solr-instance-003/./collection1/data/]
[junit4] 2> 1330297 INFO (coreLoadExecutor-1215-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=17, maxMergeAtOnceExplicit=46, maxMergedSegmentMB=1.6875, floorSegmentMB=1.9091796875, forceMergeDeletesPctAllowed=10.911964320562646, segmentsPerTier=7.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6580946644050136, deletesPctAllowed=27.511140433728293
[junit4] 2> 1330622 INFO (coreLoadExecutor-1215-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1330622 INFO (coreLoadExecutor-1215-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1330623 INFO (coreLoadExecutor-1215-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=34, maxMergeAtOnceExplicit=46, maxMergedSegmentMB=53.6416015625, floorSegmentMB=1.419921875, forceMergeDeletesPctAllowed=24.052583869079925, segmentsPerTier=38.0, maxCFSSegmentSizeMB=1.8232421875, noCFSRatio=0.0, deletesPctAllowed=30.639370724224737
[junit4] 2> 1330623 INFO (coreLoadExecutor-1215-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1debd589[collection1] main]
[junit4] 2> 1330624 INFO (coreLoadExecutor-1215-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_E759039FB489C8A5-001/solr-instance-003/collection1/conf
[junit4] 2> 1330624 INFO (coreLoadExecutor-1215-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema.xml,xslt/dummy.xsl
[junit4] 2> 1330624 INFO (coreLoadExecutor-1215-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 1330658 INFO (searcherExecutor-1216-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1debd589[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1331205 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_E759039FB489C8A5-001/solr-instance-004/collection1
[junit4] 2> 1331207 WARN (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.e.j.s.AbstractConnector Ignoring deprecated socket close linger time
[junit4] 2> 1331207 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.c.s.e.JettySolrRunner Start Jetty (original configured port=0)
[junit4] 2> 1331207 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
[junit4] 2> 1331208 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.e.j.s.Server jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: c4550056e785fb5665914545889f21dc136ad9e6; jvm 1.8.0_191-b12
[junit4] 2> 1331293 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1331293 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1331293 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 1331314 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@48a52b45{/solr,null,AVAILABLE}
[junit4] 2> 1331314 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@70b63726{HTTP/1.1,[http/1.1, h2c]}{127.0.0.1:43039}
[junit4] 2> 1331314 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.e.j.s.Server Started @1331391ms
[junit4] 2> 1331314 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_E759039FB489C8A5-001/solr-instance-004/collection1/data, hostContext=/solr, hostPort=43039}
[junit4] 2> 1331314 ERROR (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1331314 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 1331315 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.1.0
[junit4] 2> 1331315 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 1331315 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1331315 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2019-04-06T12:23:24.786Z
[junit4] 2> 1331315 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_E759039FB489C8A5-001/solr-instance-004/solr.xml
[junit4] 2> 1331331 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 1331331 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 1331332 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@6d3255be, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 1332149 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
[junit4] 2> 1332151 WARN (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@b7cda16[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1332171 WARN (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@9cfa8a6[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1332343 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 1332343 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 1332541 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6d3255be
[junit4] 2> 1332616 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6d3255be
[junit4] 2> 1332617 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6d3255be
[junit4] 2> 1332635 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_E759039FB489C8A5-001/solr-instance-004/.
[junit4] 2> 1332635 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1332991 INFO (coreLoadExecutor-1228-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.1.0
[junit4] 2> 1333014 INFO (coreLoadExecutor-1228-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1333028 INFO (coreLoadExecutor-1228-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
[junit4] 2> 1333028 INFO (coreLoadExecutor-1228-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_E759039FB489C8A5-001/solr-instance-004/./collection1, trusted=true
[junit4] 2> 1333028 INFO (coreLoadExecutor-1228-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6d3255be
[junit4] 2> 1333029 INFO (coreLoadExecutor-1228-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_E759039FB489C8A5-001/solr-instance-004/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_E759039FB489C8A5-001/solr-instance-004/./collection1/data/]
[junit4] 2> 1333047 INFO (coreLoadExecutor-1228-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=17, maxMergeAtOnceExplicit=46, maxMergedSegmentMB=1.6875, floorSegmentMB=1.9091796875, forceMergeDeletesPctAllowed=10.911964320562646, segmentsPerTier=7.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6580946644050136, deletesPctAllowed=27.511140433728293
[junit4] 2> 1333231 INFO (coreLoadExecutor-1228-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1333231 INFO (coreLoadExecutor-1228-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1333232 INFO (coreLoadExecutor-1228-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=34, maxMergeAtOnceExplicit=46, maxMergedSegmentMB=53.6416015625, floorSegmentMB=1.419921875, forceMergeDeletesPctAllowed=24.052583869079925, segmentsPerTier=38.0, maxCFSSegmentSizeMB=1.8232421875, noCFSRatio=0.0, deletesPctAllowed=30.639370724224737
[junit4] 2> 1333232 INFO (coreLoadExecutor-1228-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@641f86f4[collection1] main]
[junit4] 2> 1333232 INFO (coreLoadExecutor-1228-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_E759039FB489C8A5-001/solr-instance-004/collection1/conf
[junit4] 2> 1333246 INFO (coreLoadExecutor-1228-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 1000ms
[junit4] 2> 1333246 INFO (coreLoadExecutor-1228-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 1333247 INFO (searcherExecutor-1229-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@641f86f4[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1333249 INFO (qtp1440702187-3509) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 1333253 INFO (indexFetcher-1233-thread-1) [ ] o.a.s.h.IndexFetcher Master's generation: 1
[junit4] 2> 1333253 INFO (indexFetcher-1233-thread-1) [ ] o.a.s.h.IndexFetcher Master's version: 0
[junit4] 2> 1333253 INFO (indexFetcher-1233-thread-1) [ ] o.a.s.h.IndexFetcher Slave's generation: 1
[junit4] 2> 1333253 INFO (indexFetcher-1233-thread-1) [ ] o.a.s.h.IndexFetcher Slave's version: 0
[junit4] 2> 1333253 INFO (indexFetcher-1233-thread-1) [ ] o.a.s.h.IndexFetcher New index in Master. Deleting mine...
[junit4] 2> 1333301 INFO (indexFetcher-1233-thread-1) [ ] o.a.s.s.SolrIndexSearcher Opening [Searcher@515225e9[collection1] main]
[junit4] 2> 1333305 INFO (searcherExecutor-1229-thread-1) [ ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@515225e9[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1333336 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1328738875
[junit4] 2> 1333400 INFO (coreCloseExecutor-1234-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@1427d10a
[junit4] 2> 1333400 INFO (coreCloseExecutor-1234-thread-1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=1427d10a
[junit4] 2> 1333400 INFO (coreCloseExecutor-1234-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4e80dcb8: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@4c5a9061
[junit4] 2> 1333452 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 1333453 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@29fe46f8: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@493283dc
[junit4] 2> 1333470 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 1333470 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1141b8a9: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@c8facb9
[junit4] 2> 1333486 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 1333486 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@9250fb4: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@44d56aca
[junit4] 2> 1333501 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@70b63726{HTTP/1.1,[http/1.1, h2c]}{127.0.0.1:0}
[junit4] 2> 1333502 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@48a52b45{/solr,null,UNAVAILABLE}
[junit4] 2> 1333502 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 1333538 INFO (qtp1440702187-3510) [ x:collection1] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
[junit4] 2> 1333539 INFO (qtp1440702187-3510) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{deleteByQuery=*:*} 0 1
[junit4] 2> 1333540 INFO (qtp1440702187-3509) [ x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 1333540 INFO (qtp1440702187-3509) [ x:collection1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@4131688 commitCommandVersion:0
[junit4] 2> 1333648 INFO (qtp1440702187-3509) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4be9e241[collection1] main]
[junit4] 2> 1333648 INFO (qtp1440702187-3509) [ x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 1333649 INFO (searcherExecutor-1216-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@4be9e241[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1333649 INFO (qtp1440702187-3509) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 108
[junit4] 2> 1333650 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1446125286
[junit4] 2> 1333659 INFO (coreCloseExecutor-1236-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@18b810c2
[junit4] 2> 1333659 INFO (coreCloseExecutor-1236-thread-1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=18b810c2
[junit4] 2> 1333659 INFO (coreCloseExecutor-1236-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@307099a0: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@4845e0e1
[junit4] 2> 1333700 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 1333700 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@22bbe461: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@4a7dec8c
[junit4] 2> 1333701 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 1333701 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7697afde: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@c8facb9
[junit4] 2> 1333701 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 1333701 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@270ab8ea: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@44d56aca
[junit4] 2> 1333725 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@4d51f573{HTTP/1.1,[http/1.1, h2c]}{127.0.0.1:0}
[junit4] 2> 1333725 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@43019b71{/solr,null,UNAVAILABLE}
[junit4] 2> 1333725 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 1333726 WARN (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.e.j.s.AbstractConnector Ignoring deprecated socket close linger time
[junit4] 2> 1333726 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.c.s.e.JettySolrRunner Start Jetty (original configured port=0)
[junit4] 2> 1333726 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
[junit4] 2> 1333726 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.e.j.s.Server jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: c4550056e785fb5665914545889f21dc136ad9e6; jvm 1.8.0_191-b12
[junit4] 2> 1333727 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1333727 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1333727 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 1333728 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4ae113b0{/solr,null,AVAILABLE}
[junit4] 2> 1333728 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3a1a286b{HTTP/1.1,[http/1.1, h2c]}{127.0.0.1:44150}
[junit4] 2> 1333728 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.e.j.s.Server Started @1333806ms
[junit4] 2> 1333728 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_E759039FB489C8A5-001/solr-instance-003/collection1/data, hostContext=/solr, hostPort=44150}
[junit4] 2> 1333729 ERROR (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1333729 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 1333729 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.1.0
[junit4] 2> 1333729 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 1333729 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1333729 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2019-04-06T12:23:27.200Z
[junit4] 2> 1333729 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_E759039FB489C8A5-001/solr-instance-003/solr.xml
[junit4] 2> 1333754 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 1333754 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 1333755 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@6d3255be, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 1335314 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
[junit4] 2> 1335329 WARN (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@572d89ca[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1335348 WARN (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@2e23cf20[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1335423 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 1335423 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 1335629 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6d3255be
[junit4] 2> 1335711 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6d3255be
[junit4] 2> 1335711 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6d3255be
[junit4] 2> 1335730 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_E759039FB489C8A5-001/solr-instance-003/.
[junit4] 2> 1335730 INFO (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[E759039FB489C8A5]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 1335877 INFO (coreLoadExecutor-1246-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.1.0
[junit4] 2> 1335933 INFO (coreLoadExecutor-1246-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1335935 INFO (coreLoadExecutor-1246-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
[junit4] 2> 1335935 INFO (coreLoadExecutor-1246-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_E759039FB489C8A5-001/solr-instance-003/./collection1, trusted=true
[junit4] 2> 1335985 INFO (coreLoadExecutor-1246-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6d3255be
[junit4] 2> 1335985 INFO (coreLoadExecutor-1246-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_E759039FB489C8A5-001/solr-instance-003/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_E759039FB489C8A5-001/solr-instance-003/./collection1/data/]
[junit4] 2> 1335986 INFO (coreLoadExecutor-1246-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=17, maxMergeAtOnceExplicit=46, maxMergedSegmentMB=1.6875, floorSegmentMB=1.9091796875, forceMergeDeletesPctAllowed=10.911964320562646, segmentsPerTier=7.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.6580946644050136, deletesPctAllowed=27.511140433728293
[junit4] 2> 1336480 INFO (coreLoadExecutor-1246-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1336480 INFO (coreLoadExecutor-1246-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1336480 INFO (coreLoadExecutor-1246-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=34, maxMergeAtOnceExplicit=46, maxMergedSegmentMB=53.6416015625, floorSegmentMB=1.419921875, forceMergeDeletesPctAllowed=24.052583869079925, segmentsPerTier=38.0, maxCFSSegmentSizeMB=1.8232421875, noCFSRatio=0.0, deletesPctAllowed=30.639370724224737
[junit4] 2> 1336481 INFO (coreLoadExecutor-1246-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2d5a84d9[collection1] main]
[junit4] 2> 1336481 INFO (coreLoadExecutor-1246-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_E759039FB489C8A5-001/solr-instance-003/collection1/conf
[junit4] 2> 1336482 INFO (coreLoadExecutor-1246-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema.xml
[junit4] 2> 1336482 INFO (coreLoadExecutor-1246-thread-1) [ x:collection1] o.a.s.h.ReplicationHa
[...truncated too long message...]
ce] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/null1343927454
[copy] Copying 240 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/null1343927454
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/null1343927454
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 79ms :: artifacts dl 11ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 573 minutes 32 seconds
Archiving artifacts
java.lang.InterruptedException: no matches found within 10000
at hudson.FilePath$ValidateAntFileMask.hasMatch(FilePath.java:2847)
at hudson.FilePath$ValidateAntFileMask.invoke(FilePath.java:2726)
at hudson.FilePath$ValidateAntFileMask.invoke(FilePath.java:2707)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3086)
Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to lucene2
at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
at hudson.remoting.Channel.call(Channel.java:955)
at hudson.FilePath.act(FilePath.java:1072)
at hudson.FilePath.act(FilePath.java:1061)
at hudson.FilePath.validateAntFileMask(FilePath.java:2705)
at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
at hudson.model.Run.execute(Run.java:1835)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused: hudson.FilePath$TunneledInterruptedException
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3088)
at hudson.remoting.UserRequest.perform(UserRequest.java:212)
at hudson.remoting.UserRequest.perform(UserRequest.java:54)
at hudson.remoting.Request$2.run(Request.java:369)
at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Caused: java.lang.InterruptedException: java.lang.InterruptedException: no matches found within 10000
at hudson.FilePath.act(FilePath.java:1074)
at hudson.FilePath.act(FilePath.java:1061)
at hudson.FilePath.validateAntFileMask(FilePath.java:2705)
at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:243)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
at hudson.model.Run.execute(Run.java:1835)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
[JENKINS] Lucene-Solr-NightlyTests-8.x - Build # 63 - Failure
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-8.x/63/
1 tests failed.
FAILED: org.apache.solr.uninverting.TestDocTermOrdsUninvertLimit.testTriggerUnInvertLimit
Error Message:
GC overhead limit exceeded
Stack Trace:
java.lang.OutOfMemoryError: GC overhead limit exceeded
at __randomizedtesting.SeedInfo.seed([4B1A8D1EDD50D4F2:78A8A5DAD0E70E45]:0)
at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectField.<init>(DirectPostingsFormat.java:461)
at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectFields.<init>(DirectPostingsFormat.java:132)
at org.apache.lucene.codecs.memory.DirectPostingsFormat.fieldsProducer(DirectPostingsFormat.java:116)
at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:288)
at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:368)
at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:113)
at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:83)
at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:66)
at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:58)
at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:680)
at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:81)
at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:63)
at org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:449)
at org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:376)
at org.apache.solr.uninverting.TestDocTermOrdsUninvertLimit.testTriggerUnInvertLimit(TestDocTermOrdsUninvertLimit.java:68)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
Build Log:
[...truncated 13835 lines...]
[junit4] Suite: org.apache.solr.uninverting.TestDocTermOrdsUninvertLimit
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestDocTermOrdsUninvertLimit -Dtests.method=testTriggerUnInvertLimit -Dtests.seed=4B1A8D1EDD50D4F2 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/test-data/enwiki.random.lines.txt -Dtests.locale=es-VE -Dtests.timezone=America/Winnipeg -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 69.8s J1 | TestDocTermOrdsUninvertLimit.testTriggerUnInvertLimit <<<
[junit4] > Throwable #1: java.lang.OutOfMemoryError: GC overhead limit exceeded
[junit4] > at __randomizedtesting.SeedInfo.seed([4B1A8D1EDD50D4F2:78A8A5DAD0E70E45]:0)
[junit4] > at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectField.<init>(DirectPostingsFormat.java:461)
[junit4] > at org.apache.lucene.codecs.memory.DirectPostingsFormat$DirectFields.<init>(DirectPostingsFormat.java:132)
[junit4] > at org.apache.lucene.codecs.memory.DirectPostingsFormat.fieldsProducer(DirectPostingsFormat.java:116)
[junit4] > at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:288)
[junit4] > at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:368)
[junit4] > at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:113)
[junit4] > at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:83)
[junit4] > at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:66)
[junit4] > at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:58)
[junit4] > at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:680)
[junit4] > at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:81)
[junit4] > at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:63)
[junit4] > at org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:449)
[junit4] > at org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:376)
[junit4] > at org.apache.solr.uninverting.TestDocTermOrdsUninvertLimit.testTriggerUnInvertLimit(TestDocTermOrdsUninvertLimit.java:68)
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene80): {field=PostingsFormat(name=Direct)}, docValues:{}, maxPointsInLeafNode=95, maxMBSortInHeap=7.3138938591151295, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@56aec670), locale=es-VE, timezone=America/Winnipeg
[junit4] 2> NOTE: Linux 4.4.0-137-generic amd64/Oracle Corporation 1.8.0_191 (64-bit)/cpus=4,threads=1,free=348123752,total=529006592
[junit4] 2> NOTE: All tests run in this JVM: [TestCursorMarkWithoutUniqueKey, TestLegacyFieldReuse, BaseCdcrDistributedZkTest, StatsReloadRaceTest, TestCoreBackup, DirectUpdateHandlerTest, TestOverriddenPrefixQueryForCustomFieldType, TestDistribDocBasedVersion, JavabinLoaderTest, SuggesterWFSTTest, TestAuthorizationFramework, TestSolrConfigHandlerCloud, LeaderElectionContextKeyTest, TestMultiValuedNumericRangeQuery, TestSolrCoreParser, CollectionPropsTest, DistribDocExpirationUpdateProcessorTest, TestStressLucene, CreateCollectionCleanupTest, TestTolerantSearch, SecurityConfHandlerTest, AddSchemaFieldsUpdateProcessorFactoryTest, BlobRepositoryCloudTest, TestNumericTokenStream, ScheduledTriggerTest, SharedFSAutoReplicaFailoverTest, WrapperMergePolicyFactoryTest, TestNonDefinedSimilarityFactory, TestSolrCloudWithHadoopAuthPlugin, TestPivotHelperCode, TestLegacyNumericRangeQueryBuilder, TestShardHandlerFactory, TestFieldCacheWithThreads, TestFieldCacheSort, TestClusterProperties, ChaosMonkeyNothingIsSafeTest, UnloadDistributedZkTest, TestDynamicLoading, HdfsDirectoryTest, TestRandomFaceting, ReturnFieldsTest, TimeRoutedAliasUpdateProcessorTest, NodeLostTriggerIntegrationTest, ProtectedTermFilterFactoryTest, TestSchemaManager, TestSolrQueryParser, JWTAuthPluginIntegrationTest, TestSchemaSimilarityResource, DirectSolrSpellCheckerTest, TestInPlaceUpdatesRequiredField, TestMacros, DistributedVersionInfoTest, TestCorePropertiesReload, DistributedFacetPivotWhiteBoxTest, TestJsonFacetRefinement, CoreAdminRequestStatusTest, TestConfigSets, HdfsCollectionsAPIDistributedZkTest, TestJsonFacetsWithNestedObjects, TestCollectionAPI, DistanceFunctionTest, TestCloudConsistency, TestNestedUpdateProcessor, LeaderFailoverAfterPartitionTest, TestLazyCores, SolrInfoBeanTest, HighlighterMaxOffsetTest, DeleteLastCustomShardedReplicaTest, TestRebalanceLeaders, SyncSliceTest, TestSolrXml, TestLuceneMatchVersion, ConcurrentCreateRoutedAliasTest, TestExclusionRuleCollectionAccess, TestExceedMaxTermLength, TestFuzzyAnalyzedSuggestions, TestUnInvertedFieldException, CollectionReloadTest, TestDocTermOrdsUninvertLimit]
[junit4] Completed [219/847 (1!)] on J1 in 70.05s, 1 test, 1 error <<< FAILURES!
[...truncated 2132 lines...]
[junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/temp/junit4-J1-20190405_041330_1946554573722389673495.sysout
[junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
[junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
[junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/heapdumps/java_pid25442.hprof ...
[junit4] Heap dump file created [582872389 bytes in 1.848 secs]
[junit4] <<< JVM J1: EOF ----
[...truncated 8956 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/build.xml:651: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-8.x/checkout/build.xml:585: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid25442.hprof
Total time: 260 minutes 21 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any