You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2018/05/20 02:22:53 UTC
[JENKINS] Lucene-Solr-Tests-master - Build # 2536 - Still Unstable
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2536/
4 tests failed.
FAILED: org.apache.solr.cloud.TestCryptoKeys.test
Error Message:
Error from server at https://127.0.0.1:33174: At least one of the node(s) specified [127.0.0.1:38357_] are not currently active in [], no action taken.
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:33174: At least one of the node(s) specified [127.0.0.1:38357_] are not currently active in [], no action taken.
at __randomizedtesting.SeedInfo.seed([949C4C95DF71A91F:1CC8734F718DC4E7]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createJettys(AbstractFullDistribZkTestBase.java:425)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:341)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:991)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestCryptoKeys
Error Message:
92 threads leaked from SUITE scope at org.apache.solr.cloud.TestCryptoKeys: 1) Thread[id=9364, name=qtp798825832-9364, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=9375, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=9340, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=9285, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=9367, name=qtp798825832-9367, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715) at java.lang.Thread.run(Thread.java:748) 6) Thread[id=9328, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 7) Thread[id=9381, name=zkConnectionManagerCallback-3309-thread-1, state=WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 8) Thread[id=9253, name=TEST-TestCryptoKeys.test-seed#[949C4C95DF71A91F]-SendThread(127.0.0.1:42749), state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1054) 9) Thread[id=9299, name=zkConnectionManagerCallback-3284-thread-1, state=WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 10) Thread[id=9319, name=qtp26341971-9319, state=RUNNABLE, group=TGRP-TestCryptoKeys] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678) at java.lang.Thread.run(Thread.java:748) 11) Thread[id=9397, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 12) Thread[id=9400, name=zkConnectionManagerCallback-3316-thread-1, state=WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 13) Thread[id=9361, name=qtp798825832-9361, state=RUNNABLE, group=TGRP-TestCryptoKeys] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132) at org.eclipse.jetty.io.ManagedSelector$$Lambda$266/4850828.run(Unknown Source) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678) at java.lang.Thread.run(Thread.java:748) 14) Thread[id=9348, name=searcherExecutor-1629-thread-1, state=WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 15) Thread[id=9301, name=Scheduler-479120694, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 16) Thread[id=9297, name=TEST-TestCryptoKeys.test-seed#[949C4C95DF71A91F]-SendThread(127.0.0.1:42749), state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1054) 17) Thread[id=9344, name=Scheduler-198885779, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 18) Thread[id=9258, name=Thread-612, state=WAITING, group=TGRP-TestCryptoKeys] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:502) at org.apache.solr.core.CloserThread.run(CoreContainer.java:1743) 19) Thread[id=9272, name=qtp996720602-9272, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715) at java.lang.Thread.run(Thread.java:748) 20) Thread[id=9248, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 21) Thread[id=9362, name=qtp798825832-9362, state=RUNNABLE, group=TGRP-TestCryptoKeys] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132) at org.eclipse.jetty.io.ManagedSelector$$Lambda$266/4850828.run(Unknown Source) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678) at java.lang.Thread.run(Thread.java:748) 22) Thread[id=9373, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 23) Thread[id=9365, name=qtp798825832-9365, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715) at java.lang.Thread.run(Thread.java:748) 24) Thread[id=9380, name=TEST-TestCryptoKeys.test-seed#[949C4C95DF71A91F]-EventThread, state=WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502) 25) Thread[id=9320, name=qtp26341971-9320, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:292) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:357) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678) at java.lang.Thread.run(Thread.java:748) 26) Thread[id=9273, name=qtp996720602-9273-acceptor-0@6a16b359-ServerConnector@64d1506b{SSL,[ssl, http/1.1]}{127.0.0.1:44582}, state=RUNNABLE, group=TGRP-TestCryptoKeys] at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:378) at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:639) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678) at java.lang.Thread.run(Thread.java:748) 27) Thread[id=9341, name=TEST-TestCryptoKeys.test-seed#[949C4C95DF71A91F]-SendThread(127.0.0.1:42749), state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063) 28) Thread[id=9236, name=qtp1267771131-9236, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:292) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:357) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678) at java.lang.Thread.run(Thread.java:748) 29) Thread[id=9241, name=qtp1267771131-9241, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:292) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:357) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678) at java.lang.Thread.run(Thread.java:748) 30) Thread[id=9343, name=zkConnectionManagerCallback-3300-thread-1, state=WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 31) Thread[id=9255, name=zkConnectionManagerCallback-3262-thread-1, state=WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 32) Thread[id=9271, name=qtp996720602-9271, state=RUNNABLE, group=TGRP-TestCryptoKeys] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132) at org.eclipse.jetty.io.ManagedSelector$$Lambda$266/4850828.run(Unknown Source) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678) at java.lang.Thread.run(Thread.java:748) 33) Thread[id=9313, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 34) Thread[id=9369, name=Scheduler-294960700, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 35) Thread[id=9263, name=zkConnectionManagerCallback-3269-thread-1, state=WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 36) Thread[id=9283, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 37) Thread[id=9254, name=TEST-TestCryptoKeys.test-seed#[949C4C95DF71A91F]-EventThread, state=WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502) 38) Thread[id=9289, name=TEST-TestCryptoKeys.test-seed#[949C4C95DF71A91F]-SendThread(127.0.0.1:42749), state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063) 39) Thread[id=9342, name=TEST-TestCryptoKeys.test-seed#[949C4C95DF71A91F]-EventThread, state=WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502) 40) Thread[id=9399, name=TEST-TestCryptoKeys.test-seed#[949C4C95DF71A91F]-EventThread, state=WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502) 41) Thread[id=9243, name=Scheduler-1728744197, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 42) Thread[id=9291, name=zkConnectionManagerCallback-3277-thread-1, state=WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 43) Thread[id=9296, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 44) Thread[id=9242, name=qtp1267771131-9242, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715) at java.lang.Thread.run(Thread.java:748) 45) Thread[id=9279, name=Scheduler-2131062326, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 46) Thread[id=9355, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 47) Thread[id=9235, name=qtp1267771131-9235, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715) at java.lang.Thread.run(Thread.java:748) 48) Thread[id=9323, name=Scheduler-1689285055, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 49) Thread[id=9411, name=OverseerCollectionConfigSetProcessor-72654951734312971-127.0.0.1:36882_-n_0000000001, state=TIMED_WAITING, group=Overseer collection creation process.] at java.lang.Thread.sleep(Native Method) at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:111) at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:70) at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:340) at org.apache.solr.cloud.OverseerTaskProcessor.amILeader(OverseerTaskProcessor.java:392) at org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:152) at java.lang.Thread.run(Thread.java:748) 50) Thread[id=9270, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 51) Thread[id=9329, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 52) Thread[id=9334, name=TEST-TestCryptoKeys.test-seed#[949C4C95DF71A91F]-EventThread, state=WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502) 53) Thread[id=9264, name=Scheduler-409575461, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 54) Thread[id=9412, name=OverseerAutoScalingTriggerThread-72654951734312971-127.0.0.1:36882_-n_0000000001, state=WAITING, group=Overseer autoscaling triggers] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:502) at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1409) at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1213) at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:340) at org.apache.solr.common.cloud.SolrZkClient$$Lambda$270/1880615767.execute(Unknown Source) at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60) at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:340) at org.apache.solr.client.solrj.impl.ZkDistribStateManager.getAutoScalingConfig(ZkDistribStateManager.java:176) at org.apache.solr.client.solrj.cloud.DistribStateManager.getAutoScalingConfig(DistribStateManager.java:83) at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:131) at java.lang.Thread.run(Thread.java:748) 55) Thread[id=9261, name=TEST-TestCryptoKeys.test-seed#[949C4C95DF71A91F]-SendThread(127.0.0.1:42749), state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063) 56) Thread[id=9374, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 57) Thread[id=9260, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 58) Thread[id=9395, name=Thread-639, state=WAITING, group=TGRP-TestCryptoKeys] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:502) at org.apache.solr.core.CloserThread.run(CoreContainer.java:1743) 59) Thread[id=9239, name=qtp1267771131-9239, state=RUNNABLE, group=TGRP-TestCryptoKeys] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678) at java.lang.Thread.run(Thread.java:748) 60) Thread[id=9274, name=qtp996720602-9274, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715) at java.lang.Thread.run(Thread.java:748) 61) Thread[id=9275, name=qtp996720602-9275, state=RUNNABLE, group=TGRP-TestCryptoKeys] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678) at java.lang.Thread.run(Thread.java:748) 62) Thread[id=9284, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 63) Thread[id=9238, name=qtp1267771131-9238, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715) at java.lang.Thread.run(Thread.java:748) 64) Thread[id=9262, name=TEST-TestCryptoKeys.test-seed#[949C4C95DF71A91F]-EventThread, state=WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502) 65) Thread[id=9247, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 66) Thread[id=9249, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 67) Thread[id=9240, name=qtp1267771131-9240, state=RUNNABLE, group=TGRP-TestCryptoKeys] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678) at java.lang.Thread.run(Thread.java:748) 68) Thread[id=9298, name=TEST-TestCryptoKeys.test-seed#[949C4C95DF71A91F]-EventThread, state=WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502) 69) Thread[id=9276, name=qtp996720602-9276, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715) at java.lang.Thread.run(Thread.java:748) 70) Thread[id=9277, name=qtp996720602-9277, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715) at java.lang.Thread.run(Thread.java:748) 71) Thread[id=9294, name=Thread-621, state=WAITING, group=TGRP-TestCryptoKeys] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:502) at org.apache.solr.core.CloserThread.run(CoreContainer.java:1743) 72) Thread[id=9303, name=searcherExecutor-1618-thread-1, state=WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 73) Thread[id=9267, name=searcherExecutor-1607-thread-1, state=WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 74) Thread[id=9322, name=qtp26341971-9322, state=RUNNABLE, group=TGRP-TestCryptoKeys] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678) at java.lang.Thread.run(Thread.java:748) 75) Thread[id=9338, name=Thread-630, state=WAITING, group=TGRP-TestCryptoKeys] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:502) at org.apache.solr.core.CloserThread.run(CoreContainer.java:1743) 76) Thread[id=9379, name=TEST-TestCryptoKeys.test-seed#[949C4C95DF71A91F]-SendThread(127.0.0.1:42749), state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1054) 77) Thread[id=9333, name=TEST-TestCryptoKeys.test-seed#[949C4C95DF71A91F]-SendThread(127.0.0.1:42749), state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.ClientCnxnSocketNIO.cleanup(ClientCnxnSocketNIO.java:230) at org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1249) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1173) 78) Thread[id=9327, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 79) Thread[id=9366, name=qtp798825832-9366, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715) at java.lang.Thread.run(Thread.java:748) 80) Thread[id=9278, name=qtp996720602-9278, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:292) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:357) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678) at java.lang.Thread.run(Thread.java:748) 81) Thread[id=9317, name=qtp26341971-9317-acceptor-0@21929e9-ServerConnector@42291d2f{SSL,[ssl, http/1.1]}{127.0.0.1:33174}, state=RUNNABLE, group=TGRP-TestCryptoKeys] at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:378) at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:639) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678) at java.lang.Thread.run(Thread.java:748) 82) Thread[id=9237, name=qtp1267771131-9237-acceptor-0@420ac588-ServerConnector@4fbba792{SSL,[ssl, http/1.1]}{127.0.0.1:36882}, state=RUNNABLE, group=TGRP-TestCryptoKeys] at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:378) at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:639) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678) at java.lang.Thread.run(Thread.java:748) 83) Thread[id=9316, name=qtp26341971-9316, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715) at java.lang.Thread.run(Thread.java:748) 84) Thread[id=9363, name=qtp798825832-9363-acceptor-0@a0caadb-ServerConnector@23dc0a1{SSL,[ssl, http/1.1]}{127.0.0.1:38357}, state=RUNNABLE, group=TGRP-TestCryptoKeys] at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:378) at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:639) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678) at java.lang.Thread.run(Thread.java:748) 85) Thread[id=9321, name=qtp26341971-9321, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715) at java.lang.Thread.run(Thread.java:748) 86) Thread[id=9368, name=qtp798825832-9368, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715) at java.lang.Thread.run(Thread.java:748) 87) Thread[id=9290, name=TEST-TestCryptoKeys.test-seed#[949C4C95DF71A91F]-EventThread, state=WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502) 88) Thread[id=9398, name=TEST-TestCryptoKeys.test-seed#[949C4C95DF71A91F]-SendThread(127.0.0.1:42749), state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063) 89) Thread[id=9315, name=qtp26341971-9315, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715) at java.lang.Thread.run(Thread.java:748) 90) Thread[id=9318, name=qtp26341971-9318, state=TIMED_WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715) at java.lang.Thread.run(Thread.java:748) 91) Thread[id=9410, name=OverseerStateUpdate-72654951734312971-127.0.0.1:36882_-n_0000000001, state=TIMED_WAITING, group=Overseer state updater.] at java.lang.Thread.sleep(Native Method) at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:111) at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:70) at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:340) at org.apache.solr.cloud.Overseer$ClusterStateUpdater.amILeader(Overseer.java:417) at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:145) at java.lang.Thread.run(Thread.java:748) 92) Thread[id=9335, name=zkConnectionManagerCallback-3293-thread-1, state=WAITING, group=TGRP-TestCryptoKeys] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 92 threads leaked from SUITE scope at org.apache.solr.cloud.TestCryptoKeys:
1) Thread[id=9364, name=qtp798825832-9364, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=9375, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=9340, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
4) Thread[id=9285, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
5) Thread[id=9367, name=qtp798825832-9367, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715)
at java.lang.Thread.run(Thread.java:748)
6) Thread[id=9328, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
7) Thread[id=9381, name=zkConnectionManagerCallback-3309-thread-1, state=WAITING, group=TGRP-TestCryptoKeys]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
8) Thread[id=9253, name=TEST-TestCryptoKeys.test-seed#[949C4C95DF71A91F]-SendThread(127.0.0.1:42749), state=TIMED_WAITING, group=TGRP-TestCryptoKeys]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1054)
9) Thread[id=9299, name=zkConnectionManagerCallback-3284-thread-1, state=WAITING, group=TGRP-TestCryptoKeys]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
10) Thread[id=9319, name=qtp26341971-9319, state=RUNNABLE, group=TGRP-TestCryptoKeys]
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)
at java.lang.Thread.run(Thread.java:748)
11) Thread[id=9397, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
12) Thread[id=9400, name=zkConnectionManagerCallback-3316-thread-1, state=WAITING, group=TGRP-TestCryptoKeys]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
13) Thread[id=9361, name=qtp798825832-9361, state=RUNNABLE, group=TGRP-TestCryptoKeys]
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
at org.eclipse.jetty.io.ManagedSelector$$Lambda$266/4850828.run(Unknown Source)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)
at java.lang.Thread.run(Thread.java:748)
14) Thread[id=9348, name=searcherExecutor-1629-thread-1, state=WAITING, group=TGRP-TestCryptoKeys]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
15) Thread[id=9301, name=Scheduler-479120694, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
16) Thread[id=9297, name=TEST-TestCryptoKeys.test-seed#[949C4C95DF71A91F]-SendThread(127.0.0.1:42749), state=TIMED_WAITING, group=TGRP-TestCryptoKeys]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1054)
17) Thread[id=9344, name=Scheduler-198885779, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
18) Thread[id=9258, name=Thread-612, state=WAITING, group=TGRP-TestCryptoKeys]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:502)
at org.apache.solr.core.CloserThread.run(CoreContainer.java:1743)
19) Thread[id=9272, name=qtp996720602-9272, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715)
at java.lang.Thread.run(Thread.java:748)
20) Thread[id=9248, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
21) Thread[id=9362, name=qtp798825832-9362, state=RUNNABLE, group=TGRP-TestCryptoKeys]
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
at org.eclipse.jetty.io.ManagedSelector$$Lambda$266/4850828.run(Unknown Source)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)
at java.lang.Thread.run(Thread.java:748)
22) Thread[id=9373, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
23) Thread[id=9365, name=qtp798825832-9365, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:652)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:48)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:715)
at java.lang.Thread.run(Thread.java:748)
24) Thread[id=9380, name=TEST-TestCryptoKeys.test-seed#[949C4C95DF71A91F]-EventThread, state=WAITING, group=TGRP-TestCryptoKeys]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
25) Thread[id=9320, name=qtp26341971-9320, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:292)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:357)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)
at java.lang.Thread.run(Thread.java:748)
26) Thread[id=9273, name=qtp996720602-9273-acceptor-0@6a16b359-ServerConnector@64d1506b{SSL,[ssl, http/1.1]}{127.0.0.1:44582}, state=RUNNABLE, group=TGRP-TestCryptoKeys]
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:378)
at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:639)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)
at java.lang.Thread.run(Thread.java:748)
27) Thread[id=9341, name=TEST-TestCryptoKeys.test-seed#[949C4C95DF71A91F]-SendThread(127.0.0.1:42749), state=TIMED_WAITING, group=TGRP-TestCryptoKeys]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
28) Thread[id=9236, name=qtp1267771131-9236, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:292)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:357)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)
at java.lang.Thread.run(Thread.java:748)
29) Thread[id=9241, name=qtp1267771131-9241, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:292)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:357)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)
at java.lang.Thread.run(Thread.java:748)
30) Thread[id=9343, name=zkConnectionManagerCallback-3300-thread-1, state=WAITING, group=TGRP-TestCryptoKeys]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
31) Thread[id=9255, name=zkConnectionManagerCallback-3262-thread-1, state=WAITING, group=TGRP-TestCryptoKeys]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
32) Thread[id=9271, name=qtp996720602-9271, state=RUNNABLE, group=TGRP-TestCryptoKeys]
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:396)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:333)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:357)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:181)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:132)
at org.eclipse.jetty.io.ManagedSelector$$Lambda$266/4850828.run(Unknown Source)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:760)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:678)
at java.lang.Thread.run(Thread.java:748)
33) Thread[id=9313, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestCryptoKeys]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(T
[...truncated too long message...]
ttings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/null582407327
[copy] Copying 39 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/null582407327
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/null582407327
resolve-example:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
resolve-server:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/null991243315
[copy] Copying 246 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/null991243315
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/null991243315
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 43ms :: artifacts dl 1ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 245 minutes 54 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
[JENKINS] Lucene-Solr-Tests-master - Build # 2539 - Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2539/
3 tests failed.
FAILED: org.apache.solr.cloud.autoscaling.NodeAddedTriggerTest.testRestoreState
Error Message:
Did not expect the processor to fire on first run! event={ "id":"607d3cc2c6345dT7jtpv4rq0cqe2on812a27fpae", "source":"node_added_trigger", "eventTime":27159297683502173, "eventType":"NODEADDED", "properties":{ "eventTimes":[27159297683502173], "nodeNames":["127.0.0.1:39659_solr"]}}
Stack Trace:
java.lang.AssertionError: Did not expect the processor to fire on first run! event={
"id":"607d3cc2c6345dT7jtpv4rq0cqe2on812a27fpae",
"source":"node_added_trigger",
"eventTime":27159297683502173,
"eventType":"NODEADDED",
"properties":{
"eventTimes":[27159297683502173],
"nodeNames":["127.0.0.1:39659_solr"]}}
at __randomizedtesting.SeedInfo.seed([860FB951B6B7BD4C:48A11DC24E8EC55A]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.autoscaling.NodeAddedTriggerTest.lambda$new$0(NodeAddedTriggerTest.java:49)
at org.apache.solr.cloud.autoscaling.NodeAddedTrigger.run(NodeAddedTrigger.java:161)
at org.apache.solr.cloud.autoscaling.NodeAddedTriggerTest.testRestoreState(NodeAddedTriggerTest.java:257)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMergeIntegration
Error Message:
did not finish processing in time
Stack Trace:
java.lang.AssertionError: did not finish processing in time
at __randomizedtesting.SeedInfo.seed([860FB951B6B7BD4C:D5B6FBE154A628B6]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.IndexSizeTriggerTest.testMergeIntegration(IndexSizeTriggerTest.java:425)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.update.SoftAutoCommitTest.testSoftCommitWithinAndHardCommitMaxTimeMixedAdds
Error Message:
soft529 wasn't fast enough
Stack Trace:
java.lang.AssertionError: soft529 wasn't fast enough
at __randomizedtesting.SeedInfo.seed([860FB951B6B7BD4C:2BAC05BEB62AAFD9]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at org.apache.solr.update.SoftAutoCommitTest.doTestSoftAndHardCommitMaxTimeMixedAdds(SoftAutoCommitTest.java:209)
at org.apache.solr.update.SoftAutoCommitTest.testSoftCommitWithinAndHardCommitMaxTimeMixedAdds(SoftAutoCommitTest.java:171)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12611 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.NodeAddedTriggerTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.NodeAddedTriggerTest_860FB951B6B7BD4C-001/init-core-data-001
[junit4] 2> 227763 INFO (SUITE-NodeAddedTriggerTest-seed#[860FB951B6B7BD4C]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 227768 INFO (SUITE-NodeAddedTriggerTest-seed#[860FB951B6B7BD4C]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 227769 INFO (SUITE-NodeAddedTriggerTest-seed#[860FB951B6B7BD4C]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 227769 INFO (SUITE-NodeAddedTriggerTest-seed#[860FB951B6B7BD4C]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.NodeAddedTriggerTest_860FB951B6B7BD4C-001/tempDir-001
[junit4] 2> 227772 INFO (SUITE-NodeAddedTriggerTest-seed#[860FB951B6B7BD4C]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 227780 INFO (Thread-322) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 227780 INFO (Thread-322) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 227840 ERROR (Thread-322) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 227880 INFO (SUITE-NodeAddedTriggerTest-seed#[860FB951B6B7BD4C]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:58971
[junit4] 2> 227908 INFO (zkConnectionManagerCallback-195-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 227961 INFO (jetty-launcher-192-thread-1) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 228036 INFO (jetty-launcher-192-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 228036 INFO (jetty-launcher-192-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 228036 INFO (jetty-launcher-192-thread-1) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 228044 INFO (jetty-launcher-192-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@604c064e{/solr,null,AVAILABLE}
[junit4] 2> 228046 INFO (jetty-launcher-192-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@22d9351d{HTTP/1.1,[http/1.1]}{127.0.0.1:39659}
[junit4] 2> 228047 INFO (jetty-launcher-192-thread-1) [ ] o.e.j.s.Server Started @228106ms
[junit4] 2> 228047 INFO (jetty-launcher-192-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=39659}
[junit4] 2> 228047 ERROR (jetty-launcher-192-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 228047 INFO (jetty-launcher-192-thread-1) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 228047 INFO (jetty-launcher-192-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 228047 INFO (jetty-launcher-192-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 228047 INFO (jetty-launcher-192-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 228047 INFO (jetty-launcher-192-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-05-21T12:56:38.566Z
[junit4] 2> 228100 INFO (zkConnectionManagerCallback-197-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 228101 INFO (jetty-launcher-192-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 228325 INFO (jetty-launcher-192-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:58971/solr
[junit4] 2> 228372 INFO (zkConnectionManagerCallback-201-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 228408 INFO (zkConnectionManagerCallback-203-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 228584 INFO (jetty-launcher-192-thread-1) [n:127.0.0.1:39659_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 228585 INFO (jetty-launcher-192-thread-1) [n:127.0.0.1:39659_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:39659_solr
[junit4] 2> 228586 INFO (jetty-launcher-192-thread-1) [n:127.0.0.1:39659_solr ] o.a.s.c.Overseer Overseer (id=73837505618640899-127.0.0.1:39659_solr-n_0000000000) starting
[junit4] 2> 228627 INFO (zkConnectionManagerCallback-210-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 228630 INFO (jetty-launcher-192-thread-1) [n:127.0.0.1:39659_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:58971/solr ready
[junit4] 2> 228644 INFO (jetty-launcher-192-thread-1) [n:127.0.0.1:39659_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39659_solr
[junit4] 2> 228712 INFO (zkCallback-209-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 228726 INFO (zkCallback-202-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 229539 INFO (jetty-launcher-192-thread-1) [n:127.0.0.1:39659_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39659.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@428d74d2
[junit4] 2> 229636 INFO (jetty-launcher-192-thread-1) [n:127.0.0.1:39659_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39659.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@428d74d2
[junit4] 2> 229637 INFO (jetty-launcher-192-thread-1) [n:127.0.0.1:39659_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39659.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@428d74d2
[junit4] 2> 229639 INFO (jetty-launcher-192-thread-1) [n:127.0.0.1:39659_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.NodeAddedTriggerTest_860FB951B6B7BD4C-001/tempDir-001/node1/.
[junit4] 2> 229925 INFO (zkConnectionManagerCallback-212-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 229996 INFO (zkConnectionManagerCallback-217-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 229998 INFO (SUITE-NodeAddedTriggerTest-seed#[860FB951B6B7BD4C]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 230000 INFO (SUITE-NodeAddedTriggerTest-seed#[860FB951B6B7BD4C]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:58971/solr ready
[junit4] 2> 230026 INFO (TEST-NodeAddedTriggerTest.testActionLifecycle-seed#[860FB951B6B7BD4C]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testActionLifecycle
[junit4] 2> 230107 INFO (TEST-NodeAddedTriggerTest.testActionLifecycle-seed#[860FB951B6B7BD4C]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testActionLifecycle
[junit4] 2> 230122 INFO (TEST-NodeAddedTriggerTest.testRestoreState-seed#[860FB951B6B7BD4C]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testRestoreState
[junit4] 2> 230139 INFO (TEST-NodeAddedTriggerTest.testRestoreState-seed#[860FB951B6B7BD4C]) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 230378 INFO (TEST-NodeAddedTriggerTest.testRestoreState-seed#[860FB951B6B7BD4C]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 230378 INFO (TEST-NodeAddedTriggerTest.testRestoreState-seed#[860FB951B6B7BD4C]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 230378 INFO (TEST-NodeAddedTriggerTest.testRestoreState-seed#[860FB951B6B7BD4C]) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 230382 INFO (TEST-NodeAddedTriggerTest.testRestoreState-seed#[860FB951B6B7BD4C]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2d12a84e{/solr,null,AVAILABLE}
[junit4] 2> 230383 INFO (TEST-NodeAddedTriggerTest.testRestoreState-seed#[860FB951B6B7BD4C]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@756ff3a2{HTTP/1.1,[http/1.1]}{127.0.0.1:54164}
[junit4] 2> 230383 INFO (TEST-NodeAddedTriggerTest.testRestoreState-seed#[860FB951B6B7BD4C]) [ ] o.e.j.s.Server Started @230442ms
[junit4] 2> 230383 INFO (TEST-NodeAddedTriggerTest.testRestoreState-seed#[860FB951B6B7BD4C]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=54164}
[junit4] 2> 230384 ERROR (TEST-NodeAddedTriggerTest.testRestoreState-seed#[860FB951B6B7BD4C]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 230384 INFO (TEST-NodeAddedTriggerTest.testRestoreState-seed#[860FB951B6B7BD4C]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 230384 INFO (TEST-NodeAddedTriggerTest.testRestoreState-seed#[860FB951B6B7BD4C]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 230384 INFO (TEST-NodeAddedTriggerTest.testRestoreState-seed#[860FB951B6B7BD4C]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 230384 INFO (TEST-NodeAddedTriggerTest.testRestoreState-seed#[860FB951B6B7BD4C]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 230384 INFO (TEST-NodeAddedTriggerTest.testRestoreState-seed#[860FB951B6B7BD4C]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-05-21T12:56:40.903Z
[junit4] 2> 230412 INFO (zkConnectionManagerCallback-219-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 230413 INFO (TEST-NodeAddedTriggerTest.testRestoreState-seed#[860FB951B6B7BD4C]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 230470 INFO (TEST-NodeAddedTriggerTest.testRestoreState-seed#[860FB951B6B7BD4C]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:58971/solr
[junit4] 2> 230494 INFO (zkConnectionManagerCallback-223-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 230517 INFO (zkConnectionManagerCallback-225-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 230543 INFO (TEST-NodeAddedTriggerTest.testRestoreState-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:54164_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 230550 INFO (TEST-NodeAddedTriggerTest.testRestoreState-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:54164_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 230554 INFO (TEST-NodeAddedTriggerTest.testRestoreState-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:54164_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 230554 INFO (TEST-NodeAddedTriggerTest.testRestoreState-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:54164_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54164_solr
[junit4] 2> 230556 INFO (zkCallback-209-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 230556 INFO (zkCallback-202-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 230606 INFO (zkCallback-216-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 230615 INFO (zkCallback-224-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 231418 INFO (TEST-NodeAddedTriggerTest.testRestoreState-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:54164_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_54164.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@428d74d2
[junit4] 2> 231435 INFO (TEST-NodeAddedTriggerTest.testRestoreState-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:54164_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_54164.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@428d74d2
[junit4] 2> 231435 INFO (TEST-NodeAddedTriggerTest.testRestoreState-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:54164_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_54164.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@428d74d2
[junit4] 2> 231438 INFO (TEST-NodeAddedTriggerTest.testRestoreState-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:54164_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.NodeAddedTriggerTest_860FB951B6B7BD4C-001/tempDir-001/node2/.
[junit4] 2> 231481 INFO (zkConnectionManagerCallback-232-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 231483 INFO (TEST-NodeAddedTriggerTest.testRestoreState-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:54164_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 231485 INFO (TEST-NodeAddedTriggerTest.testRestoreState-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:54164_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:58971/solr ready
[junit4] 2> 231662 INFO (TEST-NodeAddedTriggerTest.testRestoreState-seed#[860FB951B6B7BD4C]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testRestoreState
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=NodeAddedTriggerTest -Dtests.method=testRestoreState -Dtests.seed=860FB951B6B7BD4C -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=vi-VN -Dtests.timezone=Antarctica/McMurdo -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] FAILURE 1.55s J1 | NodeAddedTriggerTest.testRestoreState <<<
[junit4] > Throwable #1: java.lang.AssertionError: Did not expect the processor to fire on first run! event={
[junit4] > "id":"607d3cc2c6345dT7jtpv4rq0cqe2on812a27fpae",
[junit4] > "source":"node_added_trigger",
[junit4] > "eventTime":27159297683502173,
[junit4] > "eventType":"NODEADDED",
[junit4] > "properties":{
[junit4] > "eventTimes":[27159297683502173],
[junit4] > "nodeNames":["127.0.0.1:39659_solr"]}}
[junit4] > at __randomizedtesting.SeedInfo.seed([860FB951B6B7BD4C:48A11DC24E8EC55A]:0)
[junit4] > at org.apache.solr.cloud.autoscaling.NodeAddedTriggerTest.lambda$new$0(NodeAddedTriggerTest.java:49)
[junit4] > at org.apache.solr.cloud.autoscaling.NodeAddedTrigger.run(NodeAddedTrigger.java:161)
[junit4] > at org.apache.solr.cloud.autoscaling.NodeAddedTriggerTest.testRestoreState(NodeAddedTriggerTest.java:257)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 231694 INFO (TEST-NodeAddedTriggerTest.testListenerAcceptance-seed#[860FB951B6B7BD4C]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testListenerAcceptance
[junit4] 2> 231696 INFO (TEST-NodeAddedTriggerTest.testListenerAcceptance-seed#[860FB951B6B7BD4C]) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 231711 INFO (TEST-NodeAddedTriggerTest.testListenerAcceptance-seed#[860FB951B6B7BD4C]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 231711 INFO (TEST-NodeAddedTriggerTest.testListenerAcceptance-seed#[860FB951B6B7BD4C]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 231711 INFO (TEST-NodeAddedTriggerTest.testListenerAcceptance-seed#[860FB951B6B7BD4C]) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 231712 INFO (TEST-NodeAddedTriggerTest.testListenerAcceptance-seed#[860FB951B6B7BD4C]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@740d6e77{/solr,null,AVAILABLE}
[junit4] 2> 231712 INFO (TEST-NodeAddedTriggerTest.testListenerAcceptance-seed#[860FB951B6B7BD4C]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@5eec3a25{HTTP/1.1,[http/1.1]}{127.0.0.1:56855}
[junit4] 2> 231712 INFO (TEST-NodeAddedTriggerTest.testListenerAcceptance-seed#[860FB951B6B7BD4C]) [ ] o.e.j.s.Server Started @231771ms
[junit4] 2> 231712 INFO (TEST-NodeAddedTriggerTest.testListenerAcceptance-seed#[860FB951B6B7BD4C]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=56855}
[junit4] 2> 231713 ERROR (TEST-NodeAddedTriggerTest.testListenerAcceptance-seed#[860FB951B6B7BD4C]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 231713 INFO (TEST-NodeAddedTriggerTest.testListenerAcceptance-seed#[860FB951B6B7BD4C]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 231713 INFO (TEST-NodeAddedTriggerTest.testListenerAcceptance-seed#[860FB951B6B7BD4C]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 231713 INFO (TEST-NodeAddedTriggerTest.testListenerAcceptance-seed#[860FB951B6B7BD4C]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 231713 INFO (TEST-NodeAddedTriggerTest.testListenerAcceptance-seed#[860FB951B6B7BD4C]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 231713 INFO (TEST-NodeAddedTriggerTest.testListenerAcceptance-seed#[860FB951B6B7BD4C]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-05-21T12:56:42.232Z
[junit4] 2> 231915 INFO (zkConnectionManagerCallback-234-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 231916 INFO (TEST-NodeAddedTriggerTest.testListenerAcceptance-seed#[860FB951B6B7BD4C]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 231960 INFO (TEST-NodeAddedTriggerTest.testListenerAcceptance-seed#[860FB951B6B7BD4C]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:58971/solr
[junit4] 2> 231970 INFO (zkConnectionManagerCallback-238-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 231982 INFO (zkConnectionManagerCallback-240-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 231991 INFO (TEST-NodeAddedTriggerTest.testListenerAcceptance-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:56855_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 231994 INFO (TEST-NodeAddedTriggerTest.testListenerAcceptance-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:56855_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 231997 INFO (TEST-NodeAddedTriggerTest.testListenerAcceptance-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:56855_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 231997 INFO (TEST-NodeAddedTriggerTest.testListenerAcceptance-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:56855_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56855_solr
[junit4] 2> 232002 INFO (zkCallback-224-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 232003 INFO (zkCallback-202-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 232003 INFO (zkCallback-209-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 232003 INFO (zkCallback-216-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 232038 INFO (zkCallback-231-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 232039 INFO (zkCallback-239-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 232378 INFO (TEST-NodeAddedTriggerTest.testListenerAcceptance-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:56855_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_56855.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@428d74d2
[junit4] 2> 232413 INFO (TEST-NodeAddedTriggerTest.testListenerAcceptance-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:56855_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_56855.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@428d74d2
[junit4] 2> 232414 INFO (TEST-NodeAddedTriggerTest.testListenerAcceptance-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:56855_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_56855.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@428d74d2
[junit4] 2> 232430 INFO (TEST-NodeAddedTriggerTest.testListenerAcceptance-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:56855_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.NodeAddedTriggerTest_860FB951B6B7BD4C-001/tempDir-001/node3/.
[junit4] 2> 232488 INFO (zkConnectionManagerCallback-247-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 232491 INFO (TEST-NodeAddedTriggerTest.testListenerAcceptance-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:56855_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 232493 INFO (TEST-NodeAddedTriggerTest.testListenerAcceptance-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:56855_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:58971/solr ready
[junit4] 2> 232657 INFO (TEST-NodeAddedTriggerTest.testListenerAcceptance-seed#[860FB951B6B7BD4C]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testListenerAcceptance
[junit4] 2> 232910 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testTrigger
[junit4] 2> 232916 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 232992 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 232992 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 232992 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 232996 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@209ddc17{/solr,null,AVAILABLE}
[junit4] 2> 232997 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@67c29ff3{HTTP/1.1,[http/1.1]}{127.0.0.1:44483}
[junit4] 2> 232997 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.e.j.s.Server Started @233056ms
[junit4] 2> 232997 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44483}
[junit4] 2> 232997 ERROR (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 232997 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 232997 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 232997 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 232998 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 232998 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-05-21T12:56:43.517Z
[junit4] 2> 233052 INFO (zkConnectionManagerCallback-249-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 233060 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 233147 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:58971/solr
[junit4] 2> 233209 INFO (zkConnectionManagerCallback-253-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 233276 INFO (zkConnectionManagerCallback-255-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 233287 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:44483_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 233290 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:44483_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 233293 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:44483_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 233293 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:44483_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44483_solr
[junit4] 2> 233296 INFO (zkCallback-216-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 233296 INFO (zkCallback-239-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 233296 INFO (zkCallback-224-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 233296 INFO (zkCallback-231-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 233297 INFO (zkCallback-209-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 233297 INFO (zkCallback-202-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 233337 INFO (zkCallback-254-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 233337 INFO (zkCallback-246-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 234103 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:44483_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44483.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@428d74d2
[junit4] 2> 234134 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:44483_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44483.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@428d74d2
[junit4] 2> 234134 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:44483_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44483.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@428d74d2
[junit4] 2> 234146 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:44483_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.NodeAddedTriggerTest_860FB951B6B7BD4C-001/tempDir-001/node4/.
[junit4] 2> 234224 INFO (zkConnectionManagerCallback-262-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 234226 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:44483_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 234229 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:44483_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:58971/solr ready
[junit4] 2> 234388 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 234485 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 234485 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 234485 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 234486 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5230ef7c{/solr,null,AVAILABLE}
[junit4] 2> 234487 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7cdf48a5{HTTP/1.1,[http/1.1]}{127.0.0.1:51689}
[junit4] 2> 234487 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.e.j.s.Server Started @234546ms
[junit4] 2> 234487 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=51689}
[junit4] 2> 234487 ERROR (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 234487 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 234488 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 234488 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 234488 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 234488 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-05-21T12:56:45.007Z
[junit4] 2> 234548 INFO (zkConnectionManagerCallback-264-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 234549 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 234601 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:58971/solr
[junit4] 2> 234632 INFO (zkConnectionManagerCallback-268-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 234660 INFO (zkConnectionManagerCallback-270-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 234674 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:51689_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 234677 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:51689_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 234687 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:51689_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 234687 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:51689_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51689_solr
[junit4] 2> 234690 INFO (zkCallback-216-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 234690 INFO (zkCallback-239-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 234691 INFO (zkCallback-224-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 234691 INFO (zkCallback-209-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 234691 INFO (zkCallback-231-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 234691 INFO (zkCallback-202-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 234711 INFO (zkCallback-261-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 234712 INFO (zkCallback-246-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 234712 INFO (zkCallback-269-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 234712 INFO (zkCallback-254-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 236043 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:51689_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51689.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@428d74d2
[junit4] 2> 236093 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:51689_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51689.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@428d74d2
[junit4] 2> 236093 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:51689_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51689.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@428d74d2
[junit4] 2> 236095 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:51689_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.NodeAddedTriggerTest_860FB951B6B7BD4C-001/tempDir-001/node5/.
[junit4] 2> 236152 INFO (zkConnectionManagerCallback-277-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 236154 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:51689_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (5)
[junit4] 2> 236155 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:51689_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:58971/solr ready
[junit4] 2> 241396 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 241466 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 241466 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 241466 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 241480 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@158483a2{/solr,null,AVAILABLE}
[junit4] 2> 241481 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@6ca8f97c{HTTP/1.1,[http/1.1]}{127.0.0.1:60973}
[junit4] 2> 241481 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.e.j.s.Server Started @241540ms
[junit4] 2> 241481 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=60973}
[junit4] 2> 241481 ERROR (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 241481 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 241481 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 241482 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 241482 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 241482 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-05-21T12:56:52.001Z
[junit4] 2> 241574 INFO (zkConnectionManagerCallback-279-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 241575 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 241637 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:58971/solr
[junit4] 2> 241792 INFO (zkConnectionManagerCallback-283-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 241834 INFO (zkConnectionManagerCallback-285-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 241862 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:60973_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (5)
[junit4] 2> 241868 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:60973_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 241872 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:60973_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 241872 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:60973_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60973_solr
[junit4] 2> 241875 INFO (zkCallback-254-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 241875 INFO (zkCallback-209-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 241876 INFO (zkCallback-202-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 241876 INFO (zkCallback-239-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 241876 INFO (zkCallback-216-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 241876 INFO (zkCallback-246-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 241876 INFO (zkCallback-224-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 241877 INFO (zkCallback-231-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 241909 INFO (zkCallback-269-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 241909 INFO (zkCallback-261-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 241909 INFO (zkCallback-276-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 241928 INFO (zkCallback-284-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 242585 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:60973_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_60973.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@428d74d2
[junit4] 2> 242615 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:60973_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_60973.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@428d74d2
[junit4] 2> 242615 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:60973_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_60973.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@428d74d2
[junit4] 2> 242622 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:60973_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.NodeAddedTriggerTest_860FB951B6B7BD4C-001/tempDir-001/node6/.
[junit4] 2> 242684 INFO (zkConnectionManagerCallback-292-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 242697 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:60973_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (6)
[junit4] 2> 242698 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [n:127.0.0.1:60973_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:58971/solr ready
[junit4] 2> 243235 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@6ca8f97c{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 243236 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1633972711
[junit4] 2> 243241 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 243241 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4a1a962b: rootName = solr_60973, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@7980a20
[junit4] 2> 243250 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 243250 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7bbd255c: rootName = solr_60973, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@577ff99c
[junit4] 2> 243263 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 243263 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@35e5cdbc: rootName = solr_60973, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@ef444d4
[junit4] 2> 243269 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:60973_solr
[junit4] 2> 243357 INFO (zkCallback-284-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (5)
[junit4] 2> 243357 INFO (zkCallback-231-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (5)
[junit4] 2> 243357 INFO (zkCallback-261-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (5)
[junit4] 2> 243358 INFO (zkCallback-202-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (5)
[junit4] 2> 243358 INFO (zkCallback-246-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (5)
[junit4] 2> 243358 INFO (zkCallback-209-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (5)
[junit4] 2> 243358 INFO (zkCallback-239-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (5)
[junit4] 2> 243358 INFO (zkCallback-216-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (5)
[junit4] 2> 243358 INFO (zkCallback-254-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (5)
[junit4] 2> 243359 INFO (zkCallback-276-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (5)
[junit4] 2> 243359 INFO (zkCallback-224-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (5)
[junit4] 2> 243362 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 243478 INFO (zkCallback-269-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (5)
[junit4] 2> 244977 WARN (zkCallback-291-thread-1) [ ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 244980 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@158483a2{/solr,null,UNAVAILABLE}
[junit4] 2> 244980 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 249988 INFO (TEST-NodeAddedTriggerTest.testTrigger-seed#[860FB951B6B7BD4C]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testTrigger
[junit4] 2> 250016 INFO (jetty-closer-193-thread-2) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@756ff3a2{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 250020 INFO (jetty-closer-193-thread-3) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@5eec3a25{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 250024 INFO (jetty-closer-193-thread-1) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@22d9351d{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 250035 INFO (jetty-closer-193-thread-4) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@67c29ff3{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 250035 INFO (jetty-closer-193-thread-5) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@7cdf48a5{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 250040 INFO (jetty-closer-193-thread-3) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1448738046
[junit4] 2> 250040 INFO (jetty-closer-193-thread-2) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=74360477
[junit4] 2> 250041 INFO (jetty-closer-193-thread-1) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1006659093
[junit4] 2> 250041 INFO (jetty-closer-193-thread-4) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=2143504993
[junit4] 2> 250041 INFO (jetty-closer-193-thread-5) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1503326168
[junit4] 2> 250044 INFO (jetty-closer-193-thread-4) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 250051 INFO (jetty-closer-193-thread-4) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3f10e4f3: rootName = solr_44483, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@7485bc10
[junit4] 2> 250057 INFO (jetty-closer-193-thread-5) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 250058 INFO (jetty-closer-193-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 250058 INFO (jetty-closer-193-thread-3) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 250058 INFO (jetty-closer-193-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 250060 INFO (jetty-closer-193-thread-5) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@73588cb1: rootName = solr_51689, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@bcd267f
[junit4] 2> 250060 INFO (jetty-closer-193-thread-4) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 250060 INFO (jetty-closer-193-thread-4) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1571f51d: rootName = solr_44483, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@577ff99c
[junit4] 2> 250063 INFO (jetty-closer-193-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3ab94679: rootName = solr_39659, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@1793b94d
[junit4] 2> 250063 INFO (jetty-closer-193-thread-3) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2c26e229: rootName = solr_56855, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@7893e615
[junit4] 2> 250063 INFO (jetty-closer-193-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@444a5b30: rootName = solr_54164, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@7c457ab1
[junit4] 2> 250082 INFO (jetty-closer-193-thread-4) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 250082 INFO (jetty-closer-193-thread-4) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@42c5563f: rootName = solr_44483, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@ef444d4
[junit4] 2> 250089 INFO (jetty-closer-193-thread-4) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:44483_solr
[junit4] 2> 250091 INFO (jetty-closer-193-thread-4) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 250093 INFO (zkCallback-202-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
[junit4] 2> 250093 INFO (zkCallback-254-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
[junit4] 2> 250093 INFO (zkCallback-269-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
[junit4] 2> 250093 INFO (zkCallback-231-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
[junit4] 2> 250093 INFO (zkCallback-261-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
[junit4] 2> 250093 INFO (zkCallback-246-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
[junit4] 2> 250093 INFO (zkCallback-239-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
[junit4] 2> 250094 INFO (zkCallback-224-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
[junit4] 2> 250094 INFO (zkCallback-276-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
[junit4] 2> 250094 INFO (zkCallback-209-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
[junit4] 2> 250130 INFO (jetty-closer-193-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 250130 INFO (jetty-closer-193-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4f00562f: rootName = solr_39659, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@577ff99c
[junit4] 2> 250132 INFO (jetty-closer-193-thread-5) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 250132 INFO (jetty-closer-193-thread-5) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7d712c3: rootName = solr_51689, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@577ff99c
[junit4] 2> 250137 INFO (jetty-closer-193-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 250137 INFO (jetty-closer-193-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@44d3a075: rootName = solr_54164, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@577ff99c
[junit4] 2> 250139 INFO (jetty-closer-193-thread-3) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 250139 INFO (jetty-closer-193-thread-3) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1ae9a9b8: rootName = solr_56855, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@577ff99c
[junit4] 2> 250201 INFO (jetty-closer-193-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 250201 INFO (jetty-closer-193-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@e083038: rootName = solr_39659, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@ef444d4
[junit4] 2> 250203 INFO (jetty-closer-193-thread-5) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 250203 INFO (jetty-closer-193-thread-5) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1ea92656: rootName = solr_51689, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@ef444d4
[junit4] 2> 250205 INFO (jetty-closer-193-thread-3) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 250205 INFO (jetty-closer-193-thread-3) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@74cf5341: rootName = solr_56855, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@ef444d4
[junit4] 2> 250206 INFO (jetty-closer-193-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 250206 INFO (jetty-closer-193-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@66246b3c: rootName = solr_54164, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@ef444d4
[junit4] 2> 250213 INFO (jetty-closer-193-thread-5) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:51689_solr
[junit4] 2> 250213 INFO (jetty-closer-193-thread-1) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:39659_solr
[junit4] 2> 250215 INFO (jetty-closer-193-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 250215 INFO (jetty-closer-193-thread-1) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 250216 INFO (jetty-closer-193-thread-1) [ ] o.a.s.c.Overseer Overseer (id=73837505618640899-127.0.0.1:39659_solr-n_0000000000) closing
[junit4] 2> 250216 INFO (zkCallback-202-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 250216 INFO (zkCallback-246-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 250216 INFO (zkCallback-231-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 250216 INFO (OverseerStateUpdate-73837505618640899-127.0.0.1:39659_solr-n_0000000000) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:39659_solr
[junit4] 2> 250216 INFO (zkCallback-209-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 250216 INFO (zkCallback-276-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 250216 INFO (zkCallback-239-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 250217 INFO (zkCallback-224-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 250225 INFO (jetty-closer-193-thread-4) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@209ddc17{/solr,null,UNAVAILABLE}
[junit4] 2> 250261 INFO (jetty-closer-193-thread-5) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 250261 WARN (OverseerAutoScalingTriggerThread-73837505618640899-127.0.0.1:39659_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 250262 INFO (jetty-closer-193-thread-4) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 250263 INFO (zkCallback-202-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 250263 INFO (zkCallback-246-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 250268 INFO (zkCallback-269-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (2)
[junit4] 2> 250276 INFO (zkCallback-239-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 250276 INFO (zkCallback-209-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 250276 INFO (zkCallback-231-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 250277 INFO (zkCallback-276-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 250279 INFO (zkCallback-224-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 250292 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10652d1f5800015, likely client has closed socket
[junit4] 2> 250295 INFO (jetty-closer-193-thread-3) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:56855_solr
[junit4] 2> 250295 INFO (jetty-closer-193-thread-2) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:54164_solr
[junit4] 2> 250298 INFO (jetty-closer-193-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 250298 INFO (jetty-closer-193-thread-3) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 250298 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10652d1f5800003, likely client has closed socket
[junit4] 2> 250298 INFO (zkCallback-231-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (0)
[junit4] 2> 250298 INFO (zkCallback-246-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (0)
[junit4] 2> 250299 INFO (zkCallback-239-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (0)
[junit4] 2> 250299 INFO (zkCallback-224-thread-1) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:54164_solr
[junit4] 2> 250318 INFO (jetty-closer-193-thread-5) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5230ef7c{/solr,null,UNAVAILABLE}
[junit4] 2> 250320 INFO (jetty-closer-193-thread-5) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 250321 INFO (jetty-closer-193-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@604c064e{/solr,null,UNAVAILABLE}
[junit4] 2> 250323 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10652d1f580000e, likely client has closed socket
[junit4] 2> 250324 INFO (jetty-closer-193-thread-1) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 250326 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10652d1f580000d, likely client has closed socket
[junit4] 2> 250327 INFO (jetty-closer-193-thread-3) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@740d6e77{/solr,null,UNAVAILABLE}
[junit4] 2> 250327 INFO (jetty-closer-193-thread-3) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 250331 WARN (zkCallback-224-thread-2) [ ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 250337 INFO (jetty-closer-193-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2d12a84e{/solr,null,UNAVAILABLE}
[junit4] 2> 250337 INFO (jetty-closer-193-thread-2) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 250345 ERROR (SUITE-NodeAddedTriggerTest-seed#[860FB951B6B7BD4C]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 250345 INFO (SUITE-NodeAddedTriggerTest-seed#[860FB951B6B7BD4C]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:58971 58971
[junit4] 2> 250449 INFO (Thread-322) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:58971 58971
[junit4] 2> 250450 WARN (Thread-322) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 13 /solr/aliases.json
[junit4] 2> 13 /solr/clusterprops.json
[junit4] 2> 6 /solr/security.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 13 /solr/clusterstate.json
[junit4] 2> 2 /solr/overseer_elect/election/73837505618640909-127.0.0.1:56855_solr-n_0000000002
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 13 /solr/live_nodes
[junit4] 2> 13 /solr/collections
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.NodeAddedTriggerTest_860FB951B6B7BD4C-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=372, maxMBSortInHeap=7.714399632125305, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@33c7b2ca), locale=vi-VN, timezone=Antarctica/McMurdo
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_172 (64-bit)/cpus=4,threads=1,free=231139448,total=383254528
[junit4] 2> NOTE: All tests run in this JVM: [BadCopyFieldTest, DistributedFacetPivotWhiteBoxTest, TestLeaderElectionWithEmptyReplica, SortSpecParsingTest, TestExportWriter, TestSchemaManager, TestStressCloudBlindAtomicUpdates, MetricUtilsTest, TestManagedSchemaAPI, AddBlockUpdateTest, CurrencyFieldTypeTest, WrapperMergePolicyFactoryTest, HdfsRecoverLeaseTest, NodeAddedTriggerTest]
[junit4] Completed [57/807 (1!)] on J1 in 22.85s, 4 tests, 1 failure <<< FAILURES!
[...truncated 1640 lines...]
[junit4] Suite: org.apache.solr.update.SoftAutoCommitTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.update.SoftAutoCommitTest_860FB951B6B7BD4C-001/init-core-data-001
[junit4] 2> 1957093 WARN (SUITE-SoftAutoCommitTest-seed#[860FB951B6B7BD4C]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=94 numCloses=94
[junit4] 2> 1957094 INFO (SUITE-SoftAutoCommitTest-seed#[860FB951B6B7BD4C]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 1957096 INFO (SUITE-SoftAutoCommitTest-seed#[860FB951B6B7BD4C]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 1957198 INFO (SUITE-SoftAutoCommitTest-seed#[860FB951B6B7BD4C]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 1957198 INFO (SUITE-SoftAutoCommitTest-seed#[860FB951B6B7BD4C]-worker) [ ] o.a.s.SolrTestCaseJ4 ####initCore
[junit4] 2> 1957199 INFO (SUITE-SoftAutoCommitTest-seed#[860FB951B6B7BD4C]-worker) [ ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/lib/classes]
[junit4] 2> 1957237 INFO (SUITE-SoftAutoCommitTest-seed#[860FB951B6B7BD4C]-worker) [ ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 1957297 INFO (SUITE-SoftAutoCommitTest-seed#[860FB951B6B7BD4C]-worker) [ ] o.a.s.s.IndexSchema [null] Schema name=test
[junit4] 2> 1957443 INFO (SUITE-SoftAutoCommitTest-seed#[860FB951B6B7BD4C]-worker) [ ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 1957506 INFO (SUITE-SoftAutoCommitTest-seed#[860FB951B6B7BD4C]-worker) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2249d245
[junit4] 2> 1957517 INFO (SUITE-SoftAutoCommitTest-seed#[860FB951B6B7BD4C]-worker) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2249d245
[junit4] 2> 1957517 INFO (SUITE-SoftAutoCommitTest-seed#[860FB951B6B7BD4C]-worker) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2249d245
[junit4] 2> 1957585 INFO (coreLoadExecutor-4455-thread-1) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1957586 INFO (coreLoadExecutor-4455-thread-1) [ x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/lib, /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/lib/classes]
[junit4] 2> 1957753 INFO (coreLoadExecutor-4455-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 1957783 INFO (coreLoadExecutor-4455-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 1958010 INFO (coreLoadExecutor-4455-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 1958028 INFO (coreLoadExecutor-4455-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1, trusted=true
[junit4] 2> 1958028 INFO (coreLoadExecutor-4455-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2249d245
[junit4] 2> 1958028 INFO (coreLoadExecutor-4455-thread-1) [ x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1958028 INFO (coreLoadExecutor-4455-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.update.SoftAutoCommitTest_860FB951B6B7BD4C-001/init-core-data-001/]
[junit4] 2> 1958040 INFO (coreLoadExecutor-4455-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=47, maxMergeAtOnceExplicit=20, maxMergedSegmentMB=72.6708984375, floorSegmentMB=0.5771484375, forceMergeDeletesPctAllowed=16.645500036705094, segmentsPerTier=44.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 1958236 INFO (coreLoadExecutor-4455-thread-1) [ x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1958236 INFO (coreLoadExecutor-4455-thread-1) [ x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1958237 INFO (coreLoadExecutor-4455-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1958237 INFO (coreLoadExecutor-4455-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1958239 INFO (coreLoadExecutor-4455-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=33, maxMergeAtOnceExplicit=15, maxMergedSegmentMB=1.9619140625, floorSegmentMB=1.5673828125, forceMergeDeletesPctAllowed=24.454918260096704, segmentsPerTier=12.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5885758209357294
[junit4] 2> 1958239 INFO (coreLoadExecutor-4455-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3ed00825[collection1] main]
[junit4] 2> 1958240 INFO (coreLoadExecutor-4455-thread-1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
[junit4] 2> 1958247 INFO (coreLoadExecutor-4455-thread-1) [ x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4] 2> 1958287 INFO (coreLoadExecutor-4455-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 1958287 INFO (searcherExecutor-4456-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
[junit4] 2> 1958287 INFO (searcherExecutor-4456-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell i
[...truncated too long message...]
pace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/null734097861
[copy] Copying 39 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/null734097861
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/null734097861
resolve-example:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
resolve-server:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/null1289737012
[copy] Copying 246 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/null1289737012
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/null1289737012
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 46ms :: artifacts dl 1ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 153 minutes 58 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
[JENKINS] Lucene-Solr-Tests-master - Build # 2538 - Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2538/
5 tests failed.
FAILED: org.apache.solr.cloud.TestSSLRandomization.testRandomizedSslAndClientAuth
Error Message:
Could not load collection from ZK: first_collection
Stack Trace:
org.apache.solr.common.SolrException: Could not load collection from ZK: first_collection
at __randomizedtesting.SeedInfo.seed([F7DA382E07D285A:842912E542E26926]:0)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1302)
at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:731)
at org.apache.solr.common.cloud.ClusterState.getCollectionOrNull(ClusterState.java:148)
at org.apache.solr.common.cloud.ClusterState.getCollectionOrNull(ClusterState.java:131)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:154)
at org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.checkCreateCollection(TestMiniSolrCloudClusterSSL.java:261)
at org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.checkClusterWithCollectionCreations(TestMiniSolrCloudClusterSSL.java:231)
at org.apache.solr.cloud.TestSSLRandomization.testRandomizedSslAndClientAuth(TestSSLRandomization.java:50)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/first_collection/state.json
at org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1215)
at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:340)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:340)
at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1314)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1300)
... 46 more
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestTolerantUpdateProcessorRandomCloud
Error Message:
Error uploading file /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/bad-schema-currencyfieldtype-bogus-amount-suffix.xml to zookeeper path /configs/solrCloudCollectionConfig/bad-schema-currencyfieldtype-bogus-amount-suffix.xml
Stack Trace:
java.io.IOException: Error uploading file /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/core/src/test-files/solr/collection1/conf/bad-schema-currencyfieldtype-bogus-amount-suffix.xml to zookeeper path /configs/solrCloudCollectionConfig/bad-schema-currencyfieldtype-bogus-amount-suffix.xml
at __randomizedtesting.SeedInfo.seed([F7DA382E07D285A]:0)
at org.apache.solr.common.cloud.ZkMaintenanceUtils$1.visitFile(ZkMaintenanceUtils.java:309)
at org.apache.solr.common.cloud.ZkMaintenanceUtils$1.visitFile(ZkMaintenanceUtils.java:291)
at java.nio.file.Files.walkFileTree(Files.java:2670)
at java.nio.file.Files.walkFileTree(Files.java:2742)
at org.apache.solr.common.cloud.ZkMaintenanceUtils.uploadToZK(ZkMaintenanceUtils.java:291)
at org.apache.solr.common.cloud.SolrZkClient.uploadToZK(SolrZkClient.java:793)
at org.apache.solr.common.cloud.ZkConfigManager.uploadConfigDir(ZkConfigManager.java:65)
at org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider.uploadConfig(ZkClientClusterStateProvider.java:145)
at org.apache.solr.cloud.SolrCloudTestCase$Builder.build(SolrCloudTestCase.java:201)
at org.apache.solr.cloud.SolrCloudTestCase$Builder.configure(SolrCloudTestCase.java:190)
at org.apache.solr.cloud.TestTolerantUpdateProcessorRandomCloud.createMiniSolrCloudCluster(TestTolerantUpdateProcessorRandomCloud.java:98)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:874)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /configs/solrCloudCollectionConfig/bad-schema-currencyfieldtype-bogus-amount-suffix.xml
at org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:786)
at org.apache.solr.common.cloud.SolrZkClient.lambda$makePath$8(SolrZkClient.java:496)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:495)
at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:391)
at org.apache.solr.common.cloud.ZkMaintenanceUtils$1.visitFile(ZkMaintenanceUtils.java:305)
... 34 more
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestTolerantUpdateProcessorRandomCloud
Error Message:
ObjectTracker found 73 object(s) that were not released!!! [SolrZkClient, ZkController, InternalHttpClient, InternalHttpClient, SolrZkClient, InternalHttpClient, InternalHttpClient, ZkController, SolrZkClient, SolrZkClient, InternalHttpClient, SolrZkClient, InternalHttpClient, ZkController, InternalHttpClient, InternalHttpClient, SolrZkClient, ZkController, ZkController, SolrZkClient, InternalHttpClient, SolrZkClient, SolrZkClient, SolrZkClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, SolrZkClient, InternalHttpClient, InternalHttpClient, ZkController, InternalHttpClient, InternalHttpClient, SolrZkClient, InternalHttpClient, SolrZkClient, SolrZkClient, Overseer, ZkController, InternalHttpClient, SolrZkClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, SolrZkClient, SolrZkClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, ZkController, InternalHttpClient, InternalHttpClient, ZkController, InternalHttpClient, InternalHttpClient, SolrZkClient, InternalHttpClient, InternalHttpClient, ZkController, InternalHttpClient, SolrZkClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, InternalHttpClient, SolrZkClient, SolrZkClient, InternalHttpClient, InternalHttpClient] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.common.cloud.SolrZkClient at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:185) at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:120) at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:110) at org.apache.solr.common.cloud.ZkStateReader.<init>(ZkStateReader.java:294) at org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider.connect(ZkClientClusterStateProvider.java:155) at org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider.uploadConfig(ZkClientClusterStateProvider.java:144) at org.apache.solr.cloud.SolrCloudTestCase$Builder.build(SolrCloudTestCase.java:201) at org.apache.solr.cloud.SolrCloudTestCase$Builder.configure(SolrCloudTestCase.java:190) at org.apache.solr.cloud.TestTolerantUpdateProcessorRandomCloud.createMiniSolrCloudCluster(TestTolerantUpdateProcessorRandomCloud.java:98) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:874) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54) at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.cloud.ZkController at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.cloud.ZkController.<init>(ZkController.java:450) at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:113) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:524) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:253) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:173) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:741) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1477) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1542) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1186) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1023) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:473) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:308) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:396) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:369) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:392) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:248) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:319) at org.apache.solr.update.UpdateShardHandler.<init>(UpdateShardHandler.java:100) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:514) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:253) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:173) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:741) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1477) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1542) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1186) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1023) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:473) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:308) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:396) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:369) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:392) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:248) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:319) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:328) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:266) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:253) at org.apache.solr.client.solrj.impl.CloudSolrClient.<init>(CloudSolrClient.java:280) at org.apache.solr.client.solrj.impl.CloudSolrClient$Builder.build(CloudSolrClient.java:1558) at org.apache.solr.cloud.ZkController.getSolrCloudManager(ZkController.java:619) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:695) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:253) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:173) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:741) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1477) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1542) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1186) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1023) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:473) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:308) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:396) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:369) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:392) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:248) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.common.cloud.SolrZkClient at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:185) at org.apache.solr.cloud.ZkController.<init>(ZkController.java:323) at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:113) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:524) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:253) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:173) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:741) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1477) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1542) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1186) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1023) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:473) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:308) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:396) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:369) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:392) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:248) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:319) at org.apache.solr.handler.component.HttpShardHandlerFactory.init(HttpShardHandlerFactory.java:212) at org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:47) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:508) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:253) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:173) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:741) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1477) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1542) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1186) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1023) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:473) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:308) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:396) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:369) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:392) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:248) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:319) at org.apache.solr.handler.component.HttpShardHandlerFactory.init(HttpShardHandlerFactory.java:212) at org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:47) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:508) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:253) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:173) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:741) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1477) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1542) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1186) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1023) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:473) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:308) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:396) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:369) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:392) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:248) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.cloud.ZkController at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.cloud.ZkController.<init>(ZkController.java:450) at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:113) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:524) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:253) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:173) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:741) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1477) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1542) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1186) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1023) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:473) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:308) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:396) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:369) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:392) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:248) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.common.cloud.SolrZkClient at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:185) at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:120) at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:110) at org.apache.solr.common.cloud.ZkStateReader.<init>(ZkStateReader.java:294) at org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider.connect(ZkClientClusterStateProvider.java:155) at org.apache.solr.client.solrj.impl.CloudSolrClient.getZkStateReader(CloudSolrClient.java:342) at org.apache.solr.client.solrj.impl.SolrClientCloudManager.<init>(SolrClientCloudManager.java:69) at org.apache.solr.cloud.ZkController.getSolrCloudManager(ZkController.java:620) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:695) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:253) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:173) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:741) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1477) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1542) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1186) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1023) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:473) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:308) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:396) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:369) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:392) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:248) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.common.cloud.SolrZkClient at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:185) at org.apache.solr.cloud.ZkController.<init>(ZkController.java:323) at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:113) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:524) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:253) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:173) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:741) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1477) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1542) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1186) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1023) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:473) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:308) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:396) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:369) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:392) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:248) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:319) at org.apache.solr.update.UpdateShardHandler.<init>(UpdateShardHandler.java:101) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:514) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:253) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:173) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:741) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1477) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1542) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1186) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1023) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:473) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:308) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:396) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:369) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:392) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:248) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.common.cloud.SolrZkClient at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:185) at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:120) at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:110) at org.apache.solr.common.cloud.ZkStateReader.<init>(ZkStateReader.java:294) at org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider.connect(ZkClientClusterStateProvider.java:155) at org.apache.solr.client.solrj.impl.CloudSolrClient.getZkStateReader(CloudSolrClient.java:342) at org.apache.solr.client.solrj.impl.SolrClientCloudManager.<init>(SolrClientCloudManager.java:69) at org.apache.solr.cloud.ZkController.getSolrCloudManager(ZkController.java:620) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:695) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:253) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:173) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:741) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1477) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1542) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1186) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1023) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:473) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:308) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:396) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:369) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:392) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:248) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:319) at org.apache.solr.update.UpdateShardHandler.<init>(UpdateShardHandler.java:100) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:514) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:253) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:173) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:741) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1477) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1542) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1186) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1023) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:473) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:308) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:396) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:369) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:392) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:248) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.cloud.ZkController at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.cloud.ZkController.<init>(ZkController.java:450) at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:113) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:524) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:253) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:173) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:741) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1477) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1542) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1186) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1023) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:473) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:308) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:396) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:369) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:392) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:248) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:319) at org.apache.solr.handler.component.HttpShardHandlerFactory.init(HttpShardHandlerFactory.java:212) at org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:47) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:508) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:253) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:173) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:741) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1477) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1542) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1186) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1023) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:473) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:308) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:396) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:369) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:392) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:248) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:319) at org.apache.solr.update.UpdateShardHandler.<init>(UpdateShardHandler.java:101) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:514) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:253) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:173) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:741) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1477) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1542) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1186) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1023) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:473) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:308) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:396) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:369) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:392) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:248) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.common.cloud.SolrZkClient at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:185) at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:120) at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:110) at org.apache.solr.common.cloud.ZkStateReader.<init>(ZkStateReader.java:294) at org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider.connect(ZkClientClusterStateProvider.java:155) at org.apache.solr.client.solrj.impl.CloudSolrClient.getZkStateReader(CloudSolrClient.java:342) at org.apache.solr.client.solrj.impl.SolrClientCloudManager.<init>(SolrClientCloudManager.java:69) at org.apache.solr.cloud.ZkController.getSolrCloudManager(ZkController.java:620) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:695) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:253) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:173) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:741) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1477) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1542) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1186) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1023) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:473) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:308) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:396) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:369) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:392) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:248) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.cloud.ZkController at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.cloud.ZkController.<init>(ZkController.java:450) at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:113) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:524) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:253) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:173) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:741) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1477) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1542) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1186) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1023) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:473) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:308) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:396) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:369) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:392) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:248) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.cloud.ZkController at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.cloud.ZkController.<init>(ZkController.java:450) at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:113) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:524) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:253) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:173) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:741) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1477) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1542) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1186) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1023) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:473) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:308) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:396) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:369) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:392) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:248) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.common.cloud.SolrZkClient at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:185) at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:120) at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:110) at org.apache.solr.common.cloud.ZkStateReader.<init>(ZkStateReader.java:294) at org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider.connect(ZkClientClusterStateProvider.java:155) at org.apache.solr.client.solrj.impl.CloudSolrClient.getZkStateReader(CloudSolrClient.java:342) at org.apache.solr.client.solrj.impl.SolrClientCloudManager.<init>(SolrClientCloudManager.java:69) at org.apache.solr.cloud.ZkController.getSolrCloudManager(ZkController.java:620) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:695) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:253) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:173) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:741) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1477) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1542) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1186) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1023) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:473) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:308) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:396) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:369) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:392) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:248) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:319) at org.apache.solr.update.UpdateShardHandler.<init>(UpdateShardHandler.java:101) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:514) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:253) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:173) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:741) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1477) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1542) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1186) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1023) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:473) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:308) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:396) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:369) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:392) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:248) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.common.cloud.SolrZkClient at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:185) at org.apache.solr.cloud.ZkController.<init>(ZkController.java:323) at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:113) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:524) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:253) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:173) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:741) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1477) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1542) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1186) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1023) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:473) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:308) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:396) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:369) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:392) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:248) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.common.cloud.SolrZkClient at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:185) at org.apache.solr.cloud.ZkController.<init>(ZkController.java:323) at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:113) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:524) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:253) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:173) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:741) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1477) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1542) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1186) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1023) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:473) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:308) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:396) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:369) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:392) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:248) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.common.cloud.SolrZkClient at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:185) at org.apache.solr.cloud.ZkController.<init>(ZkController.java:323) at org.apache.solr.core.ZkContainer.initZooKeeper(ZkContainer.java:113) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:524) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:253) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:173) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:741) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1477) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1542) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1186) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1023) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:473) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:308) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:396) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:369) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:392) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:248) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:319) at org.apache.solr.update.UpdateShardHandler.<init>(UpdateShardHandler.java:100) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:514) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:253) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:173) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:741) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1477) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1542) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1186) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1023) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:473) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:308) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:396) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:369) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:392) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:248) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:319) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:328) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:266) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:253) at org.apache.solr.client.solrj.impl.CloudSolrClient.<init>(CloudSolrClient.java:280) at org.apache.solr.client.solrj.impl.CloudSolrClient$Builder.build(CloudSolrClient.java:1558) at org.apache.solr.cloud.ZkController.getSolrCloudManager(ZkController.java:619) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:695) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:253) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:173) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:741) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1477) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1542) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1186) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1023) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:473) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:308) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:396) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:369) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:392) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:248) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:319) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:328) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:266) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:253) at org.apache.solr.client.solrj.impl.CloudSolrClient.<init>(CloudSolrClient.java:280) at org.apache.solr.client.solrj.impl.CloudSolrClient$Builder.build(CloudSolrClient.java:1558) at org.apache.solr.cloud.ZkController.getSolrCloudManager(ZkController.java:619) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:695) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:253) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:173) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:741) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1477) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1542) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1186) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1023) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:473) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:308) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:396) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:369) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:392) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:248) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:319) at org.apache.solr.update.UpdateShardHandler.<init>(UpdateShardHandler.java:101) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:514) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:253) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:173) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:741) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1477) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1542) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1186) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1023) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:473) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:308) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:396) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:369) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:392) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:248) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:319) at org.apache.solr.handler.component.HttpShardHandlerFactory.init(HttpShardHandlerFactory.java:212) at org.apache.solr.handler.component.ShardHandlerFactory.newInstance(ShardHandlerFactory.java:47) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:508) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:253) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:173) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:741) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1477) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1542) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1186) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1023) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:473) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:308) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:396) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:369) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:392) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:248) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.common.cloud.SolrZkClient at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:185) at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:120) at org.apache.solr.common.cloud.SolrZkClient.<init>(SolrZkClient.java:110) at org.apache.solr.common.cloud.ZkStateReader.<init>(ZkStateReader.java:294) at org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider.connect(ZkClientClusterStateProvider.java:155) at org.apache.solr.client.solrj.impl.CloudSolrClient.getZkStateReader(CloudSolrClient.java:342) at org.apache.solr.client.solrj.impl.SolrClientCloudManager.<init>(SolrClientCloudManager.java:69) at org.apache.solr.cloud.ZkController.getSolrCloudManager(ZkController.java:620) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:695) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:253) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:173) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:741) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1477) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1542) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1186) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1023) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:473) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:308) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:396) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:369) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:392) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:248) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:319) at org.apache.solr.update.UpdateShardHandler.<init>(UpdateShardHandler.java:101) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:514) at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:253) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:173) at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:139) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:741) at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1477) at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1542) at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1186) at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1023) at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:473) at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:308) at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:396) at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:369) at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:392) at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:248) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.http.impl.client.InternalHttpClient at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:319) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:328) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:266) at org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:253) at org.apache.solr.client.solrj.impl.CloudSolrClient.<init>(CloudSolrClient.java:280) at org.apache.solr.client.solrj.impl.CloudSolrClient$Builder.build(CloudSolrClient.java:1558) at org.apache.solr.cloud.ZkController.getSolrCloudManager(ZkController.java:619) at org.apache.solr.cloud.Overseer.getSolrCloudManager(Overseer.java:563) at org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.<init>(OverseerCollectionMessageHandler.java:203) at org.apache.solr.cloud.OverseerCollectionConfigSetProcessor.getOverseerMessageHandlerSelector(OverseerCollectionConfigSetProcessor.java:87) at org.apache.solr.cloud.OverseerCollectionConfigSetProcessor.<init>(OverseerCollectionConfigSetProcessor.java:70) at org.apache.solr.cloud.OverseerCollectionConfigSetProcessor.<init>(OverseerCollectionConfigSetProcessor.java:41) at org.apache.solr.cloud.Overseer.start(Overseer.java:535) at org.apache.solr.cloud.OverseerElectionContext.runLeaderProcess(ElectionContext.java:851) at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:170) at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:135) at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:307) at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:216) at org.apache.solr
[...truncated too long message...]
.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/null2073419212
[copy] Copying 39 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/null2073419212
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/null2073419212
resolve-example:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
resolve-server:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/null2079627829
[copy] Copying 246 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/null2079627829
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/null2079627829
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 57ms :: artifacts dl 1ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 241 minutes 12 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
[JENKINS] Lucene-Solr-Tests-master - Build # 2537 - Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2537/
3 tests failed.
FAILED: org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToPullReplica
Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:34218/solr/pull_replica_error_handling_test_cant_connect_to_pull_replica_shard1_replica_n1
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:34218/solr/pull_replica_error_handling_test_cant_connect_to_pull_replica_shard1_replica_n1
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:654)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:957)
at org.apache.solr.cloud.TestPullReplicaErrorHandling.assertNumDocs(TestPullReplicaErrorHandling.java:241)
at org.apache.solr.cloud.TestPullReplicaErrorHandling.assertNumDocs(TestPullReplicaErrorHandling.java:251)
at org.apache.solr.cloud.TestPullReplicaErrorHandling.testCantConnectToPullReplica(TestPullReplicaErrorHandling.java:151)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:542)
... 47 more
FAILED: org.apache.solr.cloud.autoscaling.SearchRateTriggerIntegrationTest.testDeleteNode
Error Message:
unexpected DELETENODE status: {responseHeader={status=0,QTime=72},status={state=notfound,msg=Did not find [search_rate_trigger3/208eebd0175360Tcokt7xqs43hpa22m0b038lxw5/0] in any tasks queue}}
Stack Trace:
java.lang.AssertionError: unexpected DELETENODE status: {responseHeader={status=0,QTime=72},status={state=notfound,msg=Did not find [search_rate_trigger3/208eebd0175360Tcokt7xqs43hpa22m0b038lxw5/0] in any tasks queue}}
at __randomizedtesting.SeedInfo.seed([8E7BDCA2A7E60AF5:ACE91220902C8588]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.autoscaling.SearchRateTriggerIntegrationTest.lambda$testDeleteNode$6(SearchRateTriggerIntegrationTest.java:668)
at java.util.ArrayList.forEach(ArrayList.java:1257)
at org.apache.solr.cloud.autoscaling.SearchRateTriggerIntegrationTest.testDeleteNode(SearchRateTriggerIntegrationTest.java:660)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.SearchRateTriggerIntegrationTest.testBelowSearchRate
Error Message:
[Op{action=DELETEREPLICA, hints={COLL_SHARD=[{ "first":"belowRate_collection", "second":"shard1"}], REPLICA=[core_node10]}}, Op{action=NONE, hints={SRC_NODE=[127.0.0.1:38848_solr]}}, Op{action=NONE, hints={SRC_NODE=[127.0.0.1:40876_solr]}}] expected:<2> but was:<3>
Stack Trace:
java.lang.AssertionError: [Op{action=DELETEREPLICA, hints={COLL_SHARD=[{
"first":"belowRate_collection",
"second":"shard1"}], REPLICA=[core_node10]}}, Op{action=NONE, hints={SRC_NODE=[127.0.0.1:38848_solr]}}, Op{action=NONE, hints={SRC_NODE=[127.0.0.1:40876_solr]}}] expected:<2> but was:<3>
at __randomizedtesting.SeedInfo.seed([8E7BDCA2A7E60AF5:E4D67AC2FF0E8FE6]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.apache.solr.cloud.autoscaling.SearchRateTriggerIntegrationTest.testBelowSearchRate(SearchRateTriggerIntegrationTest.java:493)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12620 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.SearchRateTriggerIntegrationTest
[junit4] 2> 284499 INFO (SUITE-SearchRateTriggerIntegrationTest-seed#[8E7BDCA2A7E60AF5]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.SearchRateTriggerIntegrationTest_8E7BDCA2A7E60AF5-001/init-core-data-001
[junit4] 2> 284501 WARN (SUITE-SearchRateTriggerIntegrationTest-seed#[8E7BDCA2A7E60AF5]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=16 numCloses=16
[junit4] 2> 284501 INFO (SUITE-SearchRateTriggerIntegrationTest-seed#[8E7BDCA2A7E60AF5]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 284502 INFO (SUITE-SearchRateTriggerIntegrationTest-seed#[8E7BDCA2A7E60AF5]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 284523 INFO (SUITE-SearchRateTriggerIntegrationTest-seed#[8E7BDCA2A7E60AF5]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 5 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.SearchRateTriggerIntegrationTest_8E7BDCA2A7E60AF5-001/tempDir-001
[junit4] 2> 284523 INFO (SUITE-SearchRateTriggerIntegrationTest-seed#[8E7BDCA2A7E60AF5]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 284552 INFO (Thread-111) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 284552 INFO (Thread-111) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 284595 ERROR (Thread-111) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 284645 INFO (SUITE-SearchRateTriggerIntegrationTest-seed#[8E7BDCA2A7E60AF5]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:39016
[junit4] 2> 284662 INFO (zkConnectionManagerCallback-126-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 284716 INFO (jetty-launcher-123-thread-1) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 284717 INFO (jetty-launcher-123-thread-2) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 284736 INFO (jetty-launcher-123-thread-3) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 284861 INFO (jetty-launcher-123-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 284861 INFO (jetty-launcher-123-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 284861 INFO (jetty-launcher-123-thread-1) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 284881 INFO (jetty-launcher-123-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@75ee93ad{/solr,null,AVAILABLE}
[junit4] 2> 284987 INFO (jetty-launcher-123-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@724e2cda{SSL,[ssl, http/1.1]}{127.0.0.1:40163}
[junit4] 2> 284987 INFO (jetty-launcher-123-thread-1) [ ] o.e.j.s.Server Started @285214ms
[junit4] 2> 284987 INFO (jetty-launcher-123-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40163}
[junit4] 2> 284987 ERROR (jetty-launcher-123-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 284987 INFO (jetty-launcher-123-thread-1) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 284987 INFO (jetty-launcher-123-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 284988 INFO (jetty-launcher-123-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 284988 INFO (jetty-launcher-123-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 284988 INFO (jetty-launcher-123-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-05-20T14:34:21.063Z
[junit4] 2> 285225 INFO (jetty-launcher-123-thread-3) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 285225 INFO (jetty-launcher-123-thread-3) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 285226 INFO (jetty-launcher-123-thread-3) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 285314 INFO (jetty-launcher-123-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 285315 INFO (jetty-launcher-123-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 285315 INFO (jetty-launcher-123-thread-2) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 285335 INFO (jetty-launcher-123-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2e175b70{/solr,null,AVAILABLE}
[junit4] 2> 285336 INFO (jetty-launcher-123-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7b16703e{SSL,[ssl, http/1.1]}{127.0.0.1:38848}
[junit4] 2> 285336 INFO (jetty-launcher-123-thread-2) [ ] o.e.j.s.Server Started @285563ms
[junit4] 2> 285336 INFO (jetty-launcher-123-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=38848}
[junit4] 2> 285338 ERROR (jetty-launcher-123-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 285338 INFO (jetty-launcher-123-thread-2) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 285338 INFO (jetty-launcher-123-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 285338 INFO (jetty-launcher-123-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 285338 INFO (jetty-launcher-123-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 285338 INFO (jetty-launcher-123-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-05-20T14:34:21.413Z
[junit4] 2> 285388 INFO (jetty-launcher-123-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@40dcae46{/solr,null,AVAILABLE}
[junit4] 2> 285445 INFO (jetty-launcher-123-thread-4) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 285445 INFO (jetty-launcher-123-thread-5) [ ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
[junit4] 2> 285460 INFO (jetty-launcher-123-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@630c8415{SSL,[ssl, http/1.1]}{127.0.0.1:46733}
[junit4] 2> 285460 INFO (jetty-launcher-123-thread-3) [ ] o.e.j.s.Server Started @285688ms
[junit4] 2> 285461 INFO (jetty-launcher-123-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=46733}
[junit4] 2> 285461 ERROR (jetty-launcher-123-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 285461 INFO (jetty-launcher-123-thread-3) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 285461 INFO (jetty-launcher-123-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 285461 INFO (jetty-launcher-123-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 285461 INFO (jetty-launcher-123-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 285461 INFO (jetty-launcher-123-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-05-20T14:34:21.536Z
[junit4] 2> 285462 INFO (zkConnectionManagerCallback-128-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 285552 INFO (jetty-launcher-123-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 285651 INFO (jetty-launcher-123-thread-5) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 285651 INFO (jetty-launcher-123-thread-5) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 285651 INFO (jetty-launcher-123-thread-5) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 285671 INFO (jetty-launcher-123-thread-5) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@79db8c25{/solr,null,AVAILABLE}
[junit4] 2> 285672 INFO (jetty-launcher-123-thread-5) [ ] o.e.j.s.AbstractConnector Started ServerConnector@70b0ccf1{SSL,[ssl, http/1.1]}{127.0.0.1:40876}
[junit4] 2> 285672 INFO (jetty-launcher-123-thread-5) [ ] o.e.j.s.Server Started @285899ms
[junit4] 2> 285672 INFO (jetty-launcher-123-thread-5) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40876}
[junit4] 2> 285673 ERROR (jetty-launcher-123-thread-5) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 285673 INFO (jetty-launcher-123-thread-5) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 285673 INFO (jetty-launcher-123-thread-5) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 285673 INFO (jetty-launcher-123-thread-5) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 285673 INFO (jetty-launcher-123-thread-5) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 285673 INFO (jetty-launcher-123-thread-5) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-05-20T14:34:21.748Z
[junit4] 2> 285744 INFO (zkConnectionManagerCallback-132-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 285754 INFO (jetty-launcher-123-thread-4) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 285754 INFO (jetty-launcher-123-thread-4) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 285754 INFO (jetty-launcher-123-thread-4) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 285781 INFO (jetty-launcher-123-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39016/solr
[junit4] 2> 285813 INFO (zkConnectionManagerCallback-130-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 285827 INFO (jetty-launcher-123-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 285827 INFO (jetty-launcher-123-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 285837 INFO (zkConnectionManagerCallback-134-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 285858 INFO (jetty-launcher-123-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@70f1600b{/solr,null,AVAILABLE}
[junit4] 2> 285858 INFO (jetty-launcher-123-thread-4) [ ] o.e.j.s.AbstractConnector Started ServerConnector@44f05aab{SSL,[ssl, http/1.1]}{127.0.0.1:32805}
[junit4] 2> 285859 INFO (jetty-launcher-123-thread-4) [ ] o.e.j.s.Server Started @286086ms
[junit4] 2> 285859 INFO (jetty-launcher-123-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=32805}
[junit4] 2> 285859 ERROR (jetty-launcher-123-thread-4) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 285859 INFO (jetty-launcher-123-thread-4) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 285859 INFO (jetty-launcher-123-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 285859 INFO (jetty-launcher-123-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 285859 INFO (jetty-launcher-123-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 285859 INFO (jetty-launcher-123-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-05-20T14:34:21.934Z
[junit4] 2> 285915 INFO (zkConnectionManagerCallback-138-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 285935 INFO (jetty-launcher-123-thread-5) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 285968 INFO (jetty-launcher-123-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39016/solr
[junit4] 2> 286045 INFO (jetty-launcher-123-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39016/solr
[junit4] 2> 286232 INFO (jetty-launcher-123-thread-5) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39016/solr
[junit4] 2> 286394 INFO (zkConnectionManagerCallback-148-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 286394 INFO (zkConnectionManagerCallback-144-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 286426 INFO (zkConnectionManagerCallback-140-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 286426 INFO (zkConnectionManagerCallback-154-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 286479 INFO (zkConnectionManagerCallback-152-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 286480 INFO (jetty-launcher-123-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 286496 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x102223b88cc0008, likely client has closed socket
[junit4] 2> 286744 INFO (jetty-launcher-123-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39016/solr
[junit4] 2> 286902 INFO (zkConnectionManagerCallback-166-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 287008 INFO (zkConnectionManagerCallback-160-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 287015 INFO (zkConnectionManagerCallback-156-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 287028 INFO (zkConnectionManagerCallback-158-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 287249 INFO (zkConnectionManagerCallback-168-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 287920 INFO (jetty-launcher-123-thread-5) [n:127.0.0.1:40876_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 287930 INFO (jetty-launcher-123-thread-3) [n:127.0.0.1:46733_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 287938 INFO (jetty-launcher-123-thread-2) [n:127.0.0.1:38848_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 287954 INFO (jetty-launcher-123-thread-5) [n:127.0.0.1:40876_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:40876_solr
[junit4] 2> 287975 INFO (jetty-launcher-123-thread-4) [n:127.0.0.1:32805_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 288003 INFO (jetty-launcher-123-thread-3) [n:127.0.0.1:46733_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46733_solr
[junit4] 2> 288011 INFO (jetty-launcher-123-thread-2) [n:127.0.0.1:38848_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38848_solr
[junit4] 2> 288029 INFO (jetty-launcher-123-thread-5) [n:127.0.0.1:40876_solr ] o.a.s.c.Overseer Overseer (id=72658183084834828-127.0.0.1:40876_solr-n_0000000000) starting
[junit4] 2> 288202 INFO (zkCallback-159-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 288202 INFO (zkCallback-155-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 288210 INFO (zkCallback-167-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 288307 INFO (zkCallback-157-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 288340 INFO (jetty-launcher-123-thread-4) [n:127.0.0.1:32805_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 288340 INFO (jetty-launcher-123-thread-4) [n:127.0.0.1:32805_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:32805_solr
[junit4] 2> 288401 INFO (zkCallback-157-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 288401 INFO (zkCallback-155-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 288416 INFO (zkCallback-167-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 288421 INFO (zkCallback-159-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 288933 INFO (zkConnectionManagerCallback-181-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 289081 INFO (jetty-launcher-123-thread-5) [n:127.0.0.1:40876_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 289152 INFO (jetty-launcher-123-thread-5) [n:127.0.0.1:40876_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:39016/solr ready
[junit4] 2> 289226 DEBUG (OverseerAutoScalingTriggerThread-72658183084834828-127.0.0.1:40876_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 289227 DEBUG (OverseerAutoScalingTriggerThread-72658183084834828-127.0.0.1:40876_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 289228 DEBUG (OverseerAutoScalingTriggerThread-72658183084834828-127.0.0.1:40876_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 289228 DEBUG (OverseerAutoScalingTriggerThread-72658183084834828-127.0.0.1:40876_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 289297 DEBUG (OverseerAutoScalingTriggerThread-72658183084834828-127.0.0.1:40876_solr-n_0000000000) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:38848_solr, 127.0.0.1:46733_solr, 127.0.0.1:32805_solr]
[junit4] 2> 289311 DEBUG (OverseerAutoScalingTriggerThread-72658183084834828-127.0.0.1:40876_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 289312 DEBUG (OverseerAutoScalingTriggerThread-72658183084834828-127.0.0.1:40876_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 289340 INFO (jetty-launcher-123-thread-5) [n:127.0.0.1:40876_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 289340 INFO (jetty-launcher-123-thread-5) [n:127.0.0.1:40876_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40876_solr
[junit4] 2> 289360 DEBUG (ScheduledTrigger-213-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 289369 INFO (zkCallback-155-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 289369 INFO (zkCallback-157-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 289372 INFO (zkCallback-167-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 289465 INFO (zkCallback-180-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 289522 INFO (zkCallback-159-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 289666 INFO (jetty-launcher-123-thread-1) [n:127.0.0.1:40163_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 289668 INFO (jetty-launcher-123-thread-1) [n:127.0.0.1:40163_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 289704 INFO (jetty-launcher-123-thread-1) [n:127.0.0.1:40163_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 289704 INFO (jetty-launcher-123-thread-1) [n:127.0.0.1:40163_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40163_solr
[junit4] 2> 289737 INFO (zkCallback-167-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 289738 INFO (zkCallback-157-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 289741 INFO (zkCallback-159-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 289754 INFO (zkCallback-155-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 289758 INFO (zkCallback-180-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 289814 INFO (zkCallback-153-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 293624 DEBUG (ScheduledTrigger-213-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 294045 INFO (jetty-launcher-123-thread-4) [n:127.0.0.1:32805_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_32805.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c6cfff7
[junit4] 2> 294279 INFO (jetty-launcher-123-thread-4) [n:127.0.0.1:32805_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_32805.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c6cfff7
[junit4] 2> 294279 INFO (jetty-launcher-123-thread-4) [n:127.0.0.1:32805_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_32805.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c6cfff7
[junit4] 2> 294280 INFO (jetty-launcher-123-thread-4) [n:127.0.0.1:32805_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.SearchRateTriggerIntegrationTest_8E7BDCA2A7E60AF5-001/tempDir-001/node5/.
[junit4] 2> 294753 INFO (zkConnectionManagerCallback-186-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 294753 DEBUG (ScheduledTrigger-213-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 294851 INFO (jetty-launcher-123-thread-4) [n:127.0.0.1:32805_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (5)
[junit4] 2> 294852 INFO (jetty-launcher-123-thread-4) [n:127.0.0.1:32805_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:39016/solr ready
[junit4] 2> 294873 INFO (jetty-launcher-123-thread-3) [n:127.0.0.1:46733_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46733.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c6cfff7
[junit4] 2> 294954 INFO (jetty-launcher-123-thread-3) [n:127.0.0.1:46733_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46733.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c6cfff7
[junit4] 2> 294954 INFO (jetty-launcher-123-thread-3) [n:127.0.0.1:46733_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46733.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c6cfff7
[junit4] 2> 294955 INFO (jetty-launcher-123-thread-3) [n:127.0.0.1:46733_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.SearchRateTriggerIntegrationTest_8E7BDCA2A7E60AF5-001/tempDir-001/node3/.
[junit4] 2> 295135 INFO (zkConnectionManagerCallback-191-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 295266 INFO (jetty-launcher-123-thread-3) [n:127.0.0.1:46733_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (5)
[junit4] 2> 295267 INFO (jetty-launcher-123-thread-3) [n:127.0.0.1:46733_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:39016/solr ready
[junit4] 2> 295413 INFO (jetty-launcher-123-thread-5) [n:127.0.0.1:40876_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40876.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c6cfff7
[junit4] 2> 295544 INFO (jetty-launcher-123-thread-5) [n:127.0.0.1:40876_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40876.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c6cfff7
[junit4] 2> 295545 INFO (jetty-launcher-123-thread-5) [n:127.0.0.1:40876_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40876.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c6cfff7
[junit4] 2> 295595 INFO (jetty-launcher-123-thread-5) [n:127.0.0.1:40876_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.SearchRateTriggerIntegrationTest_8E7BDCA2A7E60AF5-001/tempDir-001/node4/.
[junit4] 2> 295757 DEBUG (ScheduledTrigger-213-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 295816 INFO (jetty-launcher-123-thread-2) [n:127.0.0.1:38848_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38848.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c6cfff7
[junit4] 2> 295946 INFO (jetty-launcher-123-thread-2) [n:127.0.0.1:38848_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38848.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c6cfff7
[junit4] 2> 295947 INFO (jetty-launcher-123-thread-2) [n:127.0.0.1:38848_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38848.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c6cfff7
[junit4] 2> 295983 INFO (jetty-launcher-123-thread-2) [n:127.0.0.1:38848_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.SearchRateTriggerIntegrationTest_8E7BDCA2A7E60AF5-001/tempDir-001/node2/.
[junit4] 2> 296180 INFO (zkConnectionManagerCallback-196-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 296290 INFO (jetty-launcher-123-thread-2) [n:127.0.0.1:38848_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (5)
[junit4] 2> 296291 INFO (jetty-launcher-123-thread-2) [n:127.0.0.1:38848_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:39016/solr ready
[junit4] 2> 296760 DEBUG (ScheduledTrigger-213-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 297344 INFO (jetty-launcher-123-thread-1) [n:127.0.0.1:40163_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40163.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c6cfff7
[junit4] 2> 297379 INFO (jetty-launcher-123-thread-1) [n:127.0.0.1:40163_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40163.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c6cfff7
[junit4] 2> 297379 INFO (jetty-launcher-123-thread-1) [n:127.0.0.1:40163_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40163.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c6cfff7
[junit4] 2> 297380 INFO (jetty-launcher-123-thread-1) [n:127.0.0.1:40163_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.SearchRateTriggerIntegrationTest_8E7BDCA2A7E60AF5-001/tempDir-001/node1/.
[junit4] 2> 297440 INFO (zkConnectionManagerCallback-201-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 297442 INFO (jetty-launcher-123-thread-1) [n:127.0.0.1:40163_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (5)
[junit4] 2> 297442 INFO (jetty-launcher-123-thread-1) [n:127.0.0.1:40163_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:39016/solr ready
[junit4] 2> 297760 DEBUG (ScheduledTrigger-213-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 298097 INFO (zkConnectionManagerCallback-203-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 298115 INFO (zkConnectionManagerCallback-208-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 298132 INFO (SUITE-SearchRateTriggerIntegrationTest-seed#[8E7BDCA2A7E60AF5]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (5)
[junit4] 2> 298145 INFO (SUITE-SearchRateTriggerIntegrationTest-seed#[8E7BDCA2A7E60AF5]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:39016/solr ready
[junit4] 2> 298815 DEBUG (ScheduledTrigger-213-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 299375 INFO (qtp961713455-569) [n:127.0.0.1:32805_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=70
[junit4] 2> 299646 INFO (qtp581656984-545) [n:127.0.0.1:38848_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 299823 DEBUG (ScheduledTrigger-213-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 299845 INFO (qtp1715671018-525) [n:127.0.0.1:40163_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 299990 INFO (qtp341942661-557) [n:127.0.0.1:40876_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=102
[junit4] 2> 300407 INFO (qtp391138884-540) [n:127.0.0.1:46733_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 300477 DEBUG (qtp341942661-561) [n:127.0.0.1:40876_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 300478 DEBUG (zkCallback-180-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 300576 INFO (qtp341942661-561) [n:127.0.0.1:40876_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=1666
[junit4] 2> 300607 DEBUG (OverseerAutoScalingTriggerThread-72658183084834828-127.0.0.1:40876_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 300681 DEBUG (OverseerAutoScalingTriggerThread-72658183084834828-127.0.0.1:40876_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 300682 DEBUG (OverseerAutoScalingTriggerThread-72658183084834828-127.0.0.1:40876_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 300842 DEBUG (ScheduledTrigger-213-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 301640 INFO (TEST-SearchRateTriggerIntegrationTest.testAboveSearchRate-seed#[8E7BDCA2A7E60AF5]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testAboveSearchRate
[junit4] 2> 301641 INFO (TEST-SearchRateTriggerIntegrationTest.testAboveSearchRate-seed#[8E7BDCA2A7E60AF5]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (5)
[junit4] 2> 301660 DEBUG (zkCallback-180-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 301660 DEBUG (OverseerAutoScalingTriggerThread-72658183084834828-127.0.0.1:40876_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 301661 DEBUG (OverseerAutoScalingTriggerThread-72658183084834828-127.0.0.1:40876_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 301674 INFO (TEST-SearchRateTriggerIntegrationTest.testAboveSearchRate-seed#[8E7BDCA2A7E60AF5]) [ ] o.a.s.c.a.SearchRateTriggerIntegrationTest /autoscaling.json reset, new znode version 3
[junit4] 2> 301677 DEBUG (OverseerAutoScalingTriggerThread-72658183084834828-127.0.0.1:40876_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 301678 DEBUG (OverseerAutoScalingTriggerThread-72658183084834828-127.0.0.1:40876_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 306685 INFO (qtp581656984-548) [n:127.0.0.1:38848_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf&name=aboveRate_collection&nrtReplicas=2&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 306699 INFO (OverseerThreadFactory-215-thread-1) [ ] o.a.s.c.a.c.CreateCollectionCmd Create collection aboveRate_collection
[junit4] 2> 306860 INFO (OverseerStateUpdate-72658183084834828-127.0.0.1:40876_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"aboveRate_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"aboveRate_collection_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:32805/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 306876 INFO (OverseerStateUpdate-72658183084834828-127.0.0.1:40876_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"aboveRate_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"aboveRate_collection_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:40876/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 307160 INFO (qtp961713455-574) [n:127.0.0.1:32805_solr x:aboveRate_collection_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=aboveRate_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=aboveRate_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 307237 INFO (qtp341942661-561) [n:127.0.0.1:40876_solr x:aboveRate_collection_shard1_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=aboveRate_collection_shard1_replica_n2&action=CREATE&numShards=1&collection=aboveRate_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 308240 INFO (qtp961713455-574) [n:127.0.0.1:32805_solr c:aboveRate_collection s:shard1 r:core_node3 x:aboveRate_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 308278 INFO (qtp961713455-574) [n:127.0.0.1:32805_solr c:aboveRate_collection s:shard1 r:core_node3 x:aboveRate_collection_shard1_replica_n1] o.a.s.s.IndexSchema [aboveRate_collection_shard1_replica_n1] Schema name=minimal
[junit4] 2> 308296 INFO (qtp961713455-574) [n:127.0.0.1:32805_solr c:aboveRate_collection s:shard1 r:core_node3 x:aboveRate_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 308296 INFO (qtp961713455-574) [n:127.0.0.1:32805_solr c:aboveRate_collection s:shard1 r:core_node3 x:aboveRate_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'aboveRate_collection_shard1_replica_n1' using configuration from collection aboveRate_collection, trusted=true
[junit4] 2> 308309 INFO (qtp961713455-574) [n:127.0.0.1:32805_solr c:aboveRate_collection s:shard1 r:core_node3 x:aboveRate_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_32805.solr.core.aboveRate_collection.shard1.replica_n1' (registry 'solr.core.aboveRate_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c6cfff7
[junit4] 2> 308309 INFO (qtp961713455-574) [n:127.0.0.1:32805_solr c:aboveRate_collection s:shard1 r:core_node3 x:aboveRate_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 308309 INFO (qtp961713455-574) [n:127.0.0.1:32805_solr c:aboveRate_collection s:shard1 r:core_node3 x:aboveRate_collection_shard1_replica_n1] o.a.s.c.SolrCore [[aboveRate_collection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.SearchRateTriggerIntegrationTest_8E7BDCA2A7E60AF5-001/tempDir-001/node5/aboveRate_collection_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.SearchRateTriggerIntegrationTest_8E7BDCA2A7E60AF5-001/tempDir-001/node5/./aboveRate_collection_shard1_replica_n1/data/]
[junit4] 2> 308549 INFO (qtp341942661-561) [n:127.0.0.1:40876_solr c:aboveRate_collection s:shard1 r:core_node4 x:aboveRate_collection_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 308589 INFO (qtp341942661-561) [n:127.0.0.1:40876_solr c:aboveRate_collection s:shard1 r:core_node4 x:aboveRate_collection_shard1_replica_n2] o.a.s.s.IndexSchema [aboveRate_collection_shard1_replica_n2] Schema name=minimal
[junit4] 2> 308591 INFO (qtp341942661-561) [n:127.0.0.1:40876_solr c:aboveRate_collection s:shard1 r:core_node4 x:aboveRate_collection_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 308592 INFO (qtp341942661-561) [n:127.0.0.1:40876_solr c:aboveRate_collection s:shard1 r:core_node4 x:aboveRate_collection_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'aboveRate_collection_shard1_replica_n2' using configuration from collection aboveRate_collection, trusted=true
[junit4] 2> 308592 INFO (qtp341942661-561) [n:127.0.0.1:40876_solr c:aboveRate_collection s:shard1 r:core_node4 x:aboveRate_collection_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40876.solr.core.aboveRate_collection.shard1.replica_n2' (registry 'solr.core.aboveRate_collection.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c6cfff7
[junit4] 2> 308592 INFO (qtp341942661-561) [n:127.0.0.1:40876_solr c:aboveRate_collection s:shard1 r:core_node4 x:aboveRate_collection_shard1_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 308592 INFO (qtp341942661-561) [n:127.0.0.1:40876_solr c:aboveRate_collection s:shard1 r:core_node4 x:aboveRate_collection_shard1_replica_n2] o.a.s.c.SolrCore [[aboveRate_collection_shard1_replica_n2] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.SearchRateTriggerIntegrationTest_8E7BDCA2A7E60AF5-001/tempDir-001/node4/aboveRate_collection_shard1_replica_n2], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.SearchRateTriggerIntegrationTest_8E7BDCA2A7E60AF5-001/tempDir-001/node4/./aboveRate_collection_shard1_replica_n2/data/]
[junit4] 2> 308723 INFO (qtp961713455-574) [n:127.0.0.1:32805_solr c:aboveRate_collection s:shard1 r:core_node3 x:aboveRate_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 308723 INFO (qtp961713455-574) [n:127.0.0.1:32805_solr c:aboveRate_collection s:shard1 r:core_node3 x:aboveRate_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 308725 INFO (qtp961713455-574) [n:127.0.0.1:32805_solr c:aboveRate_collection s:shard1 r:core_node3 x:aboveRate_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 308725 INFO (qtp961713455-574) [n:127.0.0.1:32805_solr c:aboveRate_collection s:shard1 r:core_node3 x:aboveRate_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 308727 INFO (qtp961713455-574) [n:127.0.0.1:32805_solr c:aboveRate_collection s:shard1 r:core_node3 x:aboveRate_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@74b7c0e4[aboveRate_collection_shard1_replica_n1] main]
[junit4] 2> 308740 INFO (qtp961713455-574) [n:127.0.0.1:32805_solr c:aboveRate_collection s:shard1 r:core_node3 x:aboveRate_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 308740 INFO (qtp961713455-574) [n:127.0.0.1:32805_solr c:aboveRate_collection s:shard1 r:core_node3 x:aboveRate_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 308741 INFO (qtp961713455-574) [n:127.0.0.1:32805_solr c:aboveRate_collection s:shard1 r:core_node3 x:aboveRate_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 308752 INFO (searcherExecutor-226-thread-1-processing-n:127.0.0.1:32805_solr x:aboveRate_collection_shard1_replica_n1 c:aboveRate_collection s:shard1 r:core_node3) [n:127.0.0.1:32805_solr c:aboveRate_collection s:shard1 r:core_node3 x:aboveRate_collection_shard1_replica_n1] o.a.s.c.SolrCore [aboveRate_collection_shard1_replica_n1] Registered new searcher Searcher@74b7c0e4[aboveRate_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 308752 INFO (qtp961713455-574) [n:127.0.0.1:32805_solr c:aboveRate_collection s:shard1 r:core_node3 x:aboveRate_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1600994027584356352
[junit4] 2> 308790 INFO (qtp961713455-574) [n:127.0.0.1:32805_solr c:aboveRate_collection s:shard1 r:core_node3 x:aboveRate_collection_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/aboveRate_collection/terms/shard1 to Terms{values={core_node3=0}, version=0}
[junit4] 2> 308881 INFO (qtp961713455-574) [n:127.0.0.1:32805_solr c:aboveRate_collection s:shard1 r:core_node3 x:aboveRate_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9930ms
[junit4] 2> 308934 INFO (qtp341942661-561) [n:127.0.0.1:40876_solr c:aboveRate_collection s:shard1 r:core_node4 x:aboveRate_collection_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 308934 INFO (qtp341942661-561) [n:127.0.0.1:40876_solr c:aboveRate_collection s:shard1 r:core_node4 x:aboveRate_collection_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 308936 INFO (qtp341942661-561) [n:127.0.0.1:40876_solr c:aboveRate_collection s:shard1 r:core_node4 x:aboveRate_collection_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 308936 INFO (qtp341942661-561) [n:127.0.0.1:40876_solr c:aboveRate_collection s:shard1 r:core_node4 x:aboveRate_collection_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 308938 INFO (qtp341942661-561) [n:127.0.0.1:40876_solr c:aboveRate_collection s:shard1 r:core_node4 x:aboveRate_collection_shard1_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@1bf127ce[aboveRate_collection_shard1_replica_n2] main]
[junit4] 2> 308975 INFO (qtp341942661-561) [n:127.0.0.1:40876_solr c:aboveRate_collection s:shard1 r:core_node4 x:aboveRate_collection_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 308975 INFO (qtp341942661-561) [n:127.0.0.1:40876_solr c:aboveRate_collection s:shard1 r:core_node4 x:aboveRate_collection_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 308976 INFO (qtp341942661-561) [n:127.0.0.1:40876_solr c:aboveRate_collection s:shard1 r:core_node4 x:aboveRate_collection_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 308976 INFO (qtp341942661-561) [n:127.0.0.1:40876_solr c:aboveRate_collection s:shard1 r:core_node4 x:aboveRate_collection_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1600994027819237376
[junit4] 2> 309024 INFO (searcherExecutor-228-thread-1-processing-n:127.0.0.1:40876_solr x:aboveRate_collection_shard1_replica_n2 c:aboveRate_collection s:shard1 r:core_node4) [n:127.0.0.1:40876_solr c:aboveRate_collection s:shard1 r:core_node4 x:aboveRate_collection_shard1_replica_n2] o.a.s.c.SolrCore [aboveRate_collection_shard1_replica_n2] Registered new searcher Searcher@1bf127ce[aboveRate_collection_shard1_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 309028 INFO (qtp341942661-561) [n:127.0.0.1:40876_solr c:aboveRate_collection s:shard1 r:core_node4 x:aboveRate_collection_shard1_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/aboveRate_collection/terms/shard1 to Terms{values={core_node3=0, core_node4=0}, version=1}
[junit4] 2> 309386 INFO (qtp961713455-574) [n:127.0.0.1:32805_solr c:aboveRate_collection s:shard1 r:core_node3 x:aboveRate_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 309386 INFO (qtp961713455-574) [n:127.0.0.1:32805_solr c:aboveRate_collection s:shard1 r:core_node3 x:aboveRate_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 309386 INFO (qtp961713455-574) [n:127.0.0.1:32805_solr c:aboveRate_collection s:shard1 r:core_node3 x:aboveRate_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:32805/solr/aboveRate_collection_shard1_replica_n1/
[junit4] 2> 309387 INFO (qtp961713455-574) [n:127.0.0.1:32805_solr c:aboveRate_collection s:shard1 r:core_node3 x:aboveRate_collection_shard1_replica_n1] o.a.s.u.PeerSync PeerSync: core=aboveRate_collection_shard1_replica_n1 url=https://127.0.0.1:32805/solr START replicas=[https://127.0.0.1:40876/solr/aboveRate_collection_shard1_replica_n2/] nUpdates=100
[junit4] 2> 309511 INFO (qtp341942661-562) [n:127.0.0.1:40876_solr c:aboveRate_collection s:shard1 r:core_node4 x:aboveRate_collection_shard1_replica_n2] o.a.s.c.S.Request [aboveRate_collection_shard1_replica_n2] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 309710 INFO (qtp961713455-574) [n:127.0.0.1:32805_solr c:aboveRate_collection s:shard1 r:core_node3 x:aboveRate_collection_shard1_replica_n1] o.a.s.u.PeerSync PeerSync: core=aboveRate_collection_shard1_replica_n1 url=https://127.0.0.1:32805/solr DONE. We have no versions. sync failed.
[junit4] 2> 309710 INFO (qtp961713455-574) [n:127.0.0.1:32805_solr c:aboveRate_collection s:shard1 r:core_node3 x:aboveRate_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 309710 INFO (qtp961713455-574) [n:127.0.0.1:32805_solr c:aboveRate_collection s:shard1 r:core_node3 x:aboveRate_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 309710 INFO (qtp961713455-574) [n:127.0.0.1:32805_solr c:aboveRate_collection s:shard1 r:core_node3 x:aboveRate_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 309747 INFO (qtp961713455-574) [n:127.0.0.1:32805_solr c:aboveRate_collection s:shard1 r:core_node3 x:aboveRate_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:32805/solr/aboveRate_collection_shard1_replica_n1/ shard1
[junit4] 2> 309967 INFO (qtp961713455-574) [n:127.0.0.1:32805_solr c:aboveRate_collection s:shard1 r:core_node3 x:aboveRate_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 309970 INFO (qtp961713455-574) [n:127.0.0.1:32805_solr c:aboveRate_collection s:shard1 r:core_node3 x:aboveRate_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=aboveRate_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=aboveRate_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2810
[junit4] 2> 310077 INFO (qtp341942661-561) [n:127.0.0.1:40876_solr c:aboveRate_collection s:shard1 r:core_node4 x:aboveRate_collection_shard1_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=aboveRate_collection_shard1_replica_n2&action=CREATE&numShards=1&collection=aboveRate_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2852
[junit4] 2> 310151 INFO (zkCallback-167-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/aboveRate_collection/state.json] for collection [aboveRate_collection] has occurred - updating... (live nodes size: [5])
[junit4] 2> 310151 INFO (zkCallback-159-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/aboveRate_collection/state.json] for collection [aboveRate_collection] has occurred - updating... (live nodes size: [5])
[junit4] 2> 310199 INFO (qtp581656984-548) [n:127.0.0.1:38848_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 310214 INFO (qtp581656984-548) [n:127.0.0.1:38848_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=2&collection.configName=conf&name=aboveRate_collection&nrtReplicas=2&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=3529
[junit4] 2> 310987 INFO (OverseerCollectionConfigSetProcessor-72658183084834828-127.0.0.1:40876_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 311102 INFO (qtp581656984-548) [n:127.0.0.1:38848_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 311286 INFO (qtp391138884-540) [n:127.0.0.1:46733_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 311865 INFO (qtp961713455-574) [n:127.0.0.1:32805_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.aboveRate_collection.shard1.replica_n1:INDEX.sizeInBytes} status=0 QTime=0
[junit4] 2> 311901 INFO (qtp961713455-572) [n:127.0.0.1:32805_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=33
[junit4] 2> 312159 INFO (qtp1715671018-526) [n:127.0.0.1:40163_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 312319 INFO (qtp341942661-561) [n:127.0.0.1:40876_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.aboveRate_collection.shard1.replica_n2:INDEX.sizeInBytes} status=0 QTime=0
[junit4] 2> 312333 INFO (qtp341942661-561) [n:127.0.0.1:40876_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 312334 DEBUG (qtp1715671018-527) [n:127.0.0.1:40163_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 312352 INFO (qtp1715671018-527) [n:127.0.0.1:40163_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=1827
[junit4] 2> 312661 DEBUG (zkCallback-180-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 312662 DEBUG (OverseerAutoScalingTriggerThread-72658183084834828-127.0.0.1:40876_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 312662 DEBUG (OverseerAutoScalingTriggerThread-72658183084834828-127.0.0.1:40876_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 312662 DEBUG (OverseerAutoScalingTriggerThread-72658183084834828-127.0.0.1:40876_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 312697 DEBUG (OverseerAutoScalingTriggerThread-72658183084834828-127.0.0.1:40876_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 312700 INFO (qtp581656984-546) [n:127.0.0.1:38848_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 312793 INFO (qtp391138884-542) [n:127.0.0.1:46733_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 312974 INFO (qtp961713455-571) [n:127.0.0.1:32805_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.aboveRate_collection.shard1.replica_n1:INDEX.sizeInBytes} status=0 QTime=125
[junit4] 2> 312996 INFO (qtp961713455-572) [n:127.0.0.1:32805_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=19
[junit4] 2> 313269 INFO (qtp1715671018-527) [n:127.0.0.1:40163_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 313784 INFO (qtp341942661-564) [n:127.0.0.1:40876_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.aboveRate_collection.shard1.replica_n2:INDEX.sizeInBytes} status=0 QTime=108
[junit4] 2> 313841 INFO (qtp341942661-563) [n:127.0.0.1:40876_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 313841 DEBUG (qtp581656984-545) [n:127.0.0.1:38848_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 313856 INFO (qtp581656984-545) [n:127.0.0.1:38848_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=1502
[junit4] 2> 313869 DEBUG (zkCallback-180-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
[junit4] 2> 313869 DEBUG (OverseerAutoScalingTriggerThread-72658183084834828-127.0.0.1:40876_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
[junit4] 2> 313869 DEBUG (OverseerAutoScalingTriggerThread-72658183084834828-127.0.0.1:40876_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 313870 DEBUG (OverseerAutoScalingTriggerThread-72658183084834828-127.0.0.1:40876_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 313870 DEBUG (OverseerAutoScalingTriggerThread-72658183084834828-127.0.0.1:40876_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
[junit4] 2> 313907 INFO (qtp581656984-545) [n:127.0.0.1:38848_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 313932 INFO (qtp391138884-540) [n:127.0.0.1:46733_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 313968 INFO (qtp961713455-571) [n:127.0.0.1:32805_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.aboveRate_collection.shard1.replica_n1:INDEX.sizeInBytes} status=0 QTime=0
[junit4] 2> 313989 INFO (qtp961713455-571) [n:127.0.0.1:32805_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=20
[junit4] 2> 314052 INFO (qtp1715671018-528) [n:127.0.0.1:40163_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 314076 INFO (qtp341942661-564) [n:127.0.0.1:40876_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.aboveRate_collection.shard1.replica_n2:INDEX.sizeInBytes} status=0 QTime=0
[junit4] 2> 314078 INFO (qtp341942661-563) [n:127.0.0.1:40876_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 314078 DEBUG (qtp581656984-546) [n:127.0.0.1:38848_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 314088 DEBUG (zkCallback-180-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
[junit4] 2> 314088 DEBUG (OverseerAutoScalingTriggerThread-72658183084834828-127.0.0.1:40876_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
[junit4] 2> 314088 DEBUG (OverseerAutoScalingTriggerThread-72658183084834828-127.0.0.1:40876_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 314089 DEBUG (OverseerAutoScalingTriggerThread-72658183084834828-127.0.0.1:40876_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 314089 DEBUG (OverseerAutoScalingTriggerThread-72658183084834828-127.0.0.1:40876_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
[junit4] 2> 314095 INFO (qtp581656984-546) [n:127.0.0.1:38848_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=227
[junit4] 2> 314115 INFO (qtp581656984-546) [n:127.0.0.1:38848_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 314133 INFO (qtp391138884-540) [n:127.0.0.1:46733_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 314134 INFO (qtp961713455-572) [n:127.0.0.1:32805_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.aboveRate_collection.shard1.replica_n1:INDEX.sizeInBytes} status=0 QTime=0
[junit4] 2> 314150 INFO (qtp961713455-571) [n:127.0.0.1:32805_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 314151 INFO (qtp1715671018-527) [n:127.0.0.1:40163_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 314169 INFO (qtp341942661-564) [n:127.0.0.1:40876_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.aboveRate_collection.shard1.replica_n2:INDEX.sizeInBytes} status=0 QTime=0
[junit4] 2> 314188 INFO (qtp341942661-563) [n:127.0.0.1:40876_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 314188 DEBUG (qtp581656984-545) [n:127.0.0.1:38848_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 314206 INFO (qtp581656984-545) [n:127.0.0.1:38848_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=93
[junit4] 2> 314220 DEBUG (zkCallback-180-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
[junit4] 2> 314220 DEBUG (OverseerAutoScalingTriggerThread-72658183084834828-127.0.0.1:40876_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
[junit4] 2> 314220 DEBUG (OverseerAutoScalingTriggerThread-72658183084834828-127.0.0.1:40876_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 314222 INFO (qtp341942661-561) [n:127.0.0.1:40876_solr c:aboveRate_collection s:shard1 r:core_node4 x:aboveRate_collection_shard1_replica_n2] o.a.s.c.S.Request [aboveRate_collection_shard1_replica_n2] webapp=/solr path=/select params={q=*:*&_stateVer_=aboveRate_collection:6&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 314234 DEBUG (OverseerAutoScalingTriggerThread-72658183084834828-127.0.0.1:40876_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 314234 DEBUG (OverseerAutoScalingTriggerThread-72658183084834828-127.0.0.1:40876_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
[junit4] 2> 314419 INFO (qtp961713455-572) [n:127.0.0.1:32805_solr c:aboveRate_collection s:shard1 r:core_node3 x:aboveRate_collection_shard1_replica_n1] o.a.s.c.S.Request [aboveRate_collection_shard1_replica_n1] webapp=/solr path=/select params={q=*:*&_stateVer_=aboveRate_collection:6&wt=javabin&version=2} hits=0 status=0 QTime=1
[junit4] 2> 314487 INFO (qtp341942661-561) [n:127.0.0.1:40876_solr c:aboveRate_collection s:shard1 r:core_node4 x:aboveRate_collection_shard1_replica_n2] o.a.s.c.S.Request [aboveRate_collection_shard1_replica_n2] webapp=/solr path=/select params={q=*:*&_stateVer_=aboveRate_collection:6&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 314506 INFO (qtp341942661-561) [n:127.0.0.1:40876_solr c:aboveRate_collection s:shard1 r:core_node4 x:aboveRate_collection_shard1_replica_n2] o.a.s.c.S.Request [aboveRate_collection_shard1_replica_n2] webapp=/solr path=/select params={q=*:*&_stateVer_=aboveRate_collection:6&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 314507 INFO (qtp341942661-561) [n:127.0.0.1:40876_solr c:aboveRate_collection s:shard1 r:core_node4 x:aboveRate_collection_shard1_replica_n2] o.a.s.c.S.Request [aboveRate_collection_shard1_replica_n2] webapp=/solr path=/select params={q=*:*&_stateVer_=aboveRate_collection:6&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 314508 INFO (qtp341942661-561) [n:127.0.0.1:40876_solr c:aboveRate_collection s:shard1 r:core_node4 x:aboveRate_collection_shard1_replica_n2] o.a.s.c.S.Request [aboveRate_collection_shard1_replica_n2] webapp=/solr path=/select params={q=*:*&_stateVer_=aboveRate_collection:6&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 314526 INFO (qtp961713455-574) [n:127.0.0.1:32805_solr c:aboveRate_collection s:shard1 r:core_node3 x:aboveRate_collection_shard1_replica_n1] o.a.s.c.S.Request [aboveRate_collection_shard1_replica_n1] webapp=/solr path=/select params={q=*:*&_stateVer_=aboveRate_collection:6&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 314528 INFO (qtp961713455-574) [n:127.0.0.1:32805_solr c:aboveRate_collection s:shard1 r:core_node3 x:aboveRate_collection_shard1_replica_n1] o.a.s.c.S.Request [aboveRate_collection_shard1_replica_n1] webapp=/solr path=/select params={q=*:*&_stateVer_=aboveRate_collection:6&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 314529 INFO (qtp961713455-574) [n:127.0.0.1:32805_solr c:aboveRate_collection s:shard1 r:core_node3 x:aboveRate_collection_shard1_replica_n1] o.a.s.c.S.Request [aboveRate_collection_shard1_replica_n1] webapp=/solr path=/select params={q=*:*&_stateVer_=aboveRate_collection:6&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 314563 INFO (qtp961713455-574) [n:127.0.0.1:32805_solr c:aboveRate_collection s:shard1 r:core_node3 x:aboveRate_collection_shard1_replica_n1] o.a.s.c.S.Request [aboveRate_collection_shard1_replica_n1] webapp=/solr path=/select params={q=*:*&_stateVer_=aboveRate_collection:6&wt=javabin&version=2} hits=0 status=0 QTime=20
[junit4] 2> 314573 INFO (qtp341942661-561) [n:127.0.0.1:40876_solr c:aboveRate_collection s:shard1 r:core_node4 x:aboveRate_collection_shard1_replica_n2] o.a.s.c.S.Request [aboveRate_collection_shard1_replica_n2] webapp=/solr path=/select params={q=*:*&_stateVer_=aboveRate_collection:6&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 314574 INFO (qtp341942661-561) [n:127.0.0.1:40876_solr c:aboveRate_collection s:shard1 r:core_node4 x:aboveRate_collection_shard1_replica_n2] o.a.s.c.S.Request [aboveRate_collection_shard1_replica_n2] webapp=/solr path=/select params={q=*:*&_stateVer_=aboveRate_collection:6&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 314599 INFO (qtp341942661-561) [n:127.0.0.1:40876_solr c:aboveRate_collection s:shard1 r:core_node4 x:aboveRate_collection_shard1_replica_n2] o.a.s.c.S.Request [aboveRate_collection_shard1_replica_n2] webapp=/solr path=/select params={q=*:*&_stateVer_=aboveRate_collection:6&wt=javabin&version=2} hits=0 status=0 QTime=18
[junit4] 2> 314601 INFO (qtp341942661-561) [n:127.0.0.1:40876_solr c:aboveRate_collection s:shard1 r:core_node4 x:aboveRate_collection_shard1_replica_n2] o.a.s.c.S.Request [aboveRate_collection_shard1_replica_n2] webapp=/solr path=/select params={q=*:*&_stateVer_=aboveRate_collection:6&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 314602 INFO (qtp341942661-561) [n:127.0.0.1:40876_solr c:aboveRate_collection s:shard1 r:core_node4 x:aboveRate_collection_shard1_replica_n2] o.a.s.c.S.Request [aboveRate_collection_shard1_replica_n2] webapp=/solr path=/select params={q=*:*&_stateVer_=aboveRate_collection:6&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 314616 INFO (qtp341942661-564) [n:127.0.0.1:40876_solr c:aboveRate_collection s:shard1 r:core_node4 x:aboveRate_collection_shard1_replica_n2] o.a.s.c.S.Request [aboveRate_collection_shard1_replica_n2] webapp=/solr path=/select params={q=*:*&_stateVer_=aboveRate_collection:6&wt=javabin&version=2} hits=0 status=0 QTime=12
[junit4] 2> 314617 INFO (qtp961713455-572) [n:127.0.0.1:32805_solr c:aboveRate_collection s:shard1 r:core_node3 x:aboveRate_collection_shard1_replica_n1] o.a.s.c.S.Request [aboveRate_collection_shard1_replica_n1] webapp=/solr path=/select pa
[...truncated too long message...]
gs.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/null626580831
[copy] Copying 39 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/null626580831
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/null626580831
resolve-example:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
resolve-server:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/null2146049593
[copy] Copying 246 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/null2146049593
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/null2146049593
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 49ms :: artifacts dl 33ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 304 minutes 41 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)