You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2018/02/03 12:33:18 UTC
[JENKINS] Lucene-Solr-7.x-Solaris (64bit/jdk1.8.0) - Build # 425 -
Still Unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Solaris/425/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseG1GC
7 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.ltr.feature.TestFieldLengthFeature
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.ltr.feature.TestFieldLengthFeature: 1) Thread[id=86, name=qtp1643342153-86, state=TIMED_WAITING, group=TGRP-TestFieldLengthFeature] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.ltr.feature.TestFieldLengthFeature:
1) Thread[id=86, name=qtp1643342153-86, state=TIMED_WAITING, group=TGRP-TestFieldLengthFeature]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([AF5A6D86DD7D06A7]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.ltr.feature.TestFieldLengthFeature
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=86, name=qtp1643342153-86, state=TIMED_WAITING, group=TGRP-TestFieldLengthFeature] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=86, name=qtp1643342153-86, state=TIMED_WAITING, group=TGRP-TestFieldLengthFeature]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([AF5A6D86DD7D06A7]:0)
FAILED: org.apache.solr.cloud.ReplaceNodeNoTargetTest.test
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([21E318AF226E3481:A9B727758C925979]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.cloud.ReplaceNodeNoTargetTest.test(ReplaceNodeNoTargetTest.java:92)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation
Error Message:
3 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation: 1) Thread[id=43645, name=jetty-launcher-11874-thread-1-EventThread, state=WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502) 2) Thread[id=43644, name=jetty-launcher-11874-thread-1-SendThread(127.0.0.1:57151), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063) 3) Thread[id=43580, name=jetty-launcher-11874-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037) at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328) at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277) at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323) at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105) at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41) at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244) at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44) at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61) at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:531) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 3 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:
1) Thread[id=43645, name=jetty-launcher-11874-thread-1-EventThread, state=WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
2) Thread[id=43644, name=jetty-launcher-11874-thread-1-SendThread(127.0.0.1:57151), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
3) Thread[id=43580, name=jetty-launcher-11874-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:531)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)
at __randomizedtesting.SeedInfo.seed([21E318AF226E3481]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=43644, name=jetty-launcher-11874-thread-1-SendThread(127.0.0.1:57151), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=43644, name=jetty-launcher-11874-thread-1-SendThread(127.0.0.1:57151), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
at __randomizedtesting.SeedInfo.seed([21E318AF226E3481]:0)
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testMetricTrigger
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([21E318AF226E3481:9BEF2F207D86E2CE]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at org.junit.Assert.assertNull(Assert.java:562)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testMetricTrigger(TriggerIntegrationTest.java:1575)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigReplication
Error Message:
Index: 0, Size: 0
Stack Trace:
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at __randomizedtesting.SeedInfo.seed([21E318AF226E3481:35AB43FA0169899F]:0)
at java.util.ArrayList.rangeCheck(ArrayList.java:657)
at java.util.ArrayList.get(ArrayList.java:433)
at org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigReplication(TestReplicationHandler.java:561)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12269 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest
[junit4] 2> 807920 INFO (SUITE-TriggerIntegrationTest-seed#[21E318AF226E3481]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_21E318AF226E3481-001/init-core-data-001
[junit4] 2> 807921 WARN (SUITE-TriggerIntegrationTest-seed#[21E318AF226E3481]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=2 numCloses=2
[junit4] 2> 807921 INFO (SUITE-TriggerIntegrationTest-seed#[21E318AF226E3481]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 807922 INFO (SUITE-TriggerIntegrationTest-seed#[21E318AF226E3481]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 807923 INFO (SUITE-TriggerIntegrationTest-seed#[21E318AF226E3481]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_21E318AF226E3481-001/tempDir-001
[junit4] 2> 807923 INFO (SUITE-TriggerIntegrationTest-seed#[21E318AF226E3481]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 807923 INFO (Thread-2008) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 807923 INFO (Thread-2008) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 807925 ERROR (Thread-2008) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 808023 INFO (SUITE-TriggerIntegrationTest-seed#[21E318AF226E3481]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:41487
[junit4] 2> 808025 INFO (zkConnectionManagerCallback-1590-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 808030 INFO (jetty-launcher-1587-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T20:27:37-01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 808030 INFO (jetty-launcher-1587-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T20:27:37-01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 808031 INFO (jetty-launcher-1587-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 808031 INFO (jetty-launcher-1587-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 808031 INFO (jetty-launcher-1587-thread-2) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 808031 INFO (jetty-launcher-1587-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 808031 INFO (jetty-launcher-1587-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@e934919{/solr,null,AVAILABLE}
[junit4] 2> 808031 INFO (jetty-launcher-1587-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 808032 INFO (jetty-launcher-1587-thread-1) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 808032 INFO (jetty-launcher-1587-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@48bdd2f2{HTTP/1.1,[http/1.1]}{127.0.0.1:63652}
[junit4] 2> 808032 INFO (jetty-launcher-1587-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@d1598c1{/solr,null,AVAILABLE}
[junit4] 2> 808032 INFO (jetty-launcher-1587-thread-2) [ ] o.e.j.s.Server Started @810836ms
[junit4] 2> 808032 INFO (jetty-launcher-1587-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=63652}
[junit4] 2> 808032 ERROR (jetty-launcher-1587-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 808032 INFO (jetty-launcher-1587-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@44d2ddf1{HTTP/1.1,[http/1.1]}{127.0.0.1:50267}
[junit4] 2> 808032 INFO (jetty-launcher-1587-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 808032 INFO (jetty-launcher-1587-thread-1) [ ] o.e.j.s.Server Started @810836ms
[junit4] 2> 808032 INFO (jetty-launcher-1587-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 808032 INFO (jetty-launcher-1587-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=50267}
[junit4] 2> 808032 INFO (jetty-launcher-1587-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 808032 INFO (jetty-launcher-1587-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-03T11:12:40.544Z
[junit4] 2> 808032 ERROR (jetty-launcher-1587-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 808032 INFO (jetty-launcher-1587-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 808032 INFO (jetty-launcher-1587-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 808032 INFO (jetty-launcher-1587-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 808033 INFO (jetty-launcher-1587-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-03T11:12:40.545Z
[junit4] 2> 808035 INFO (zkConnectionManagerCallback-1594-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 808036 INFO (jetty-launcher-1587-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 808037 INFO (zkConnectionManagerCallback-1592-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 808037 INFO (jetty-launcher-1587-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 808042 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10045d727400002, likely client has closed socket
[junit4] 2> 808045 INFO (jetty-launcher-1587-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41487/solr
[junit4] 2> 808045 INFO (jetty-launcher-1587-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41487/solr
[junit4] 2> 808047 INFO (zkConnectionManagerCallback-1600-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 808047 INFO (zkConnectionManagerCallback-1602-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 808051 INFO (zkConnectionManagerCallback-1604-thread-1-processing-n:127.0.0.1:63652_solr) [n:127.0.0.1:63652_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 808052 INFO (zkConnectionManagerCallback-1606-thread-1-processing-n:127.0.0.1:50267_solr) [n:127.0.0.1:50267_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 808235 INFO (jetty-launcher-1587-thread-2) [n:127.0.0.1:63652_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 808236 INFO (jetty-launcher-1587-thread-2) [n:127.0.0.1:63652_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:63652_solr
[junit4] 2> 808238 INFO (jetty-launcher-1587-thread-2) [n:127.0.0.1:63652_solr ] o.a.s.c.Overseer Overseer (id=72134384416718853-127.0.0.1:63652_solr-n_0000000000) starting
[junit4] 2> 808251 INFO (jetty-launcher-1587-thread-2) [n:127.0.0.1:63652_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63652_solr
[junit4] 2> 808393 INFO (zkCallback-1603-thread-1-processing-n:127.0.0.1:63652_solr) [n:127.0.0.1:63652_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 808393 DEBUG (OverseerAutoScalingTriggerThread-72134384416718853-127.0.0.1:63652_solr-n_0000000000) [n:127.0.0.1:63652_solr ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 808394 DEBUG (OverseerAutoScalingTriggerThread-72134384416718853-127.0.0.1:63652_solr-n_0000000000) [n:127.0.0.1:63652_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 808395 DEBUG (OverseerAutoScalingTriggerThread-72134384416718853-127.0.0.1:63652_solr-n_0000000000) [n:127.0.0.1:63652_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:63652_solr]
[junit4] 2> 808395 DEBUG (OverseerAutoScalingTriggerThread-72134384416718853-127.0.0.1:63652_solr-n_0000000000) [n:127.0.0.1:63652_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 808395 DEBUG (OverseerAutoScalingTriggerThread-72134384416718853-127.0.0.1:63652_solr-n_0000000000) [n:127.0.0.1:63652_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 808398 DEBUG (OverseerAutoScalingTriggerThread-72134384416718853-127.0.0.1:63652_solr-n_0000000000) [n:127.0.0.1:63652_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 808399 DEBUG (OverseerAutoScalingTriggerThread-72134384416718853-127.0.0.1:63652_solr-n_0000000000) [n:127.0.0.1:63652_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 808402 INFO (jetty-launcher-1587-thread-1) [n:127.0.0.1:50267_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 808404 INFO (jetty-launcher-1587-thread-1) [n:127.0.0.1:50267_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 808406 INFO (jetty-launcher-1587-thread-1) [n:127.0.0.1:50267_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50267_solr
[junit4] 2> 808423 INFO (zkCallback-1605-thread-1-processing-n:127.0.0.1:50267_solr) [n:127.0.0.1:50267_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 808423 DEBUG (ScheduledTrigger-2380-thread-1) [n:127.0.0.1:63652_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 808423 INFO (zkCallback-1603-thread-2-processing-n:127.0.0.1:63652_solr) [n:127.0.0.1:63652_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 808426 INFO (jetty-launcher-1587-thread-2) [n:127.0.0.1:63652_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_63652.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67aa6241
[junit4] 2> 808432 INFO (jetty-launcher-1587-thread-2) [n:127.0.0.1:63652_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_63652.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67aa6241
[junit4] 2> 808432 INFO (jetty-launcher-1587-thread-2) [n:127.0.0.1:63652_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_63652.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67aa6241
[junit4] 2> 808433 INFO (jetty-launcher-1587-thread-2) [n:127.0.0.1:63652_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_21E318AF226E3481-001/tempDir-001/node2/.
[junit4] 2> 808495 INFO (jetty-launcher-1587-thread-1) [n:127.0.0.1:50267_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50267.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67aa6241
[junit4] 2> 808500 INFO (jetty-launcher-1587-thread-1) [n:127.0.0.1:50267_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50267.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67aa6241
[junit4] 2> 808501 INFO (jetty-launcher-1587-thread-1) [n:127.0.0.1:50267_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50267.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67aa6241
[junit4] 2> 808501 INFO (jetty-launcher-1587-thread-1) [n:127.0.0.1:50267_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_21E318AF226E3481-001/tempDir-001/node1/.
[junit4] 2> 808524 INFO (zkConnectionManagerCallback-1612-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 808530 INFO (zkConnectionManagerCallback-1616-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 808531 INFO (SUITE-TriggerIntegrationTest-seed#[21E318AF226E3481]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 808532 INFO (SUITE-TriggerIntegrationTest-seed#[21E318AF226E3481]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41487/solr ready
[junit4] 2> 808552 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[21E318AF226E3481]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeLostTriggerRestoreState
[junit4] 2> 808553 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[21E318AF226E3481]) [ ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 2
[junit4] 2> 808557 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[21E318AF226E3481]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T20:27:37-01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 808558 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[21E318AF226E3481]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 808558 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[21E318AF226E3481]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 808558 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[21E318AF226E3481]) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 808559 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[21E318AF226E3481]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@719f430b{/solr,null,AVAILABLE}
[junit4] 2> 808559 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[21E318AF226E3481]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3a98f52e{HTTP/1.1,[http/1.1]}{127.0.0.1:63987}
[junit4] 2> 808559 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[21E318AF226E3481]) [ ] o.e.j.s.Server Started @811363ms
[junit4] 2> 808559 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[21E318AF226E3481]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=63987}
[junit4] 2> 808559 ERROR (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[21E318AF226E3481]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 808559 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[21E318AF226E3481]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 808559 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[21E318AF226E3481]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 808559 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[21E318AF226E3481]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 808559 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[21E318AF226E3481]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-03T11:12:41.071Z
[junit4] 2> 808562 DEBUG (zkCallback-1603-thread-2-processing-n:127.0.0.1:63652_solr) [n:127.0.0.1:63652_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 808562 INFO (zkConnectionManagerCallback-1618-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 808563 DEBUG (OverseerAutoScalingTriggerThread-72134384416718853-127.0.0.1:63652_solr-n_0000000000) [n:127.0.0.1:63652_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 808564 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[21E318AF226E3481]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 808571 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[21E318AF226E3481]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41487/solr
[junit4] 2> 808571 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10045d727400009, likely client has closed socket
[junit4] 2> 808582 DEBUG (OverseerAutoScalingTriggerThread-72134384416718853-127.0.0.1:63652_solr-n_0000000000) [n:127.0.0.1:63652_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 808583 INFO (zkConnectionManagerCallback-1622-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 808585 DEBUG (OverseerAutoScalingTriggerThread-72134384416718853-127.0.0.1:63652_solr-n_0000000000) [n:127.0.0.1:63652_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 808586 DEBUG (OverseerAutoScalingTriggerThread-72134384416718853-127.0.0.1:63652_solr-n_0000000000) [n:127.0.0.1:63652_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 808587 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10045d72740000a, likely client has closed socket
[junit4] 2> 808588 INFO (zkConnectionManagerCallback-1624-thread-1-processing-n:127.0.0.1:63987_solr) [n:127.0.0.1:63987_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 808596 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[21E318AF226E3481]) [n:127.0.0.1:63987_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 808599 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[21E318AF226E3481]) [n:127.0.0.1:63987_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 808606 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[21E318AF226E3481]) [n:127.0.0.1:63987_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63987_solr
[junit4] 2> 808608 INFO (zkCallback-1615-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 808609 INFO (zkCallback-1623-thread-1-processing-n:127.0.0.1:63987_solr) [n:127.0.0.1:63987_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 808610 INFO (zkCallback-1605-thread-1-processing-n:127.0.0.1:50267_solr) [n:127.0.0.1:50267_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 808610 INFO (zkCallback-1603-thread-2-processing-n:127.0.0.1:63652_solr) [n:127.0.0.1:63652_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 808712 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[21E318AF226E3481]) [n:127.0.0.1:63987_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_63987.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67aa6241
[junit4] 2> 808725 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[21E318AF226E3481]) [n:127.0.0.1:63987_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_63987.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67aa6241
[junit4] 2> 808725 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[21E318AF226E3481]) [n:127.0.0.1:63987_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_63987.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67aa6241
[junit4] 2> 808726 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[21E318AF226E3481]) [n:127.0.0.1:63987_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_21E318AF226E3481-001/tempDir-001/node3/.
[junit4] 2> 808785 INFO (qtp1300884583-7326) [n:127.0.0.1:50267_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 808788 INFO (qtp1072870045-7311) [n:127.0.0.1:63652_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 808792 INFO (qtp595415689-7372) [n:127.0.0.1:63987_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 808792 DEBUG (qtp1300884583-7321) [n:127.0.0.1:50267_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 808795 INFO (qtp1300884583-7321) [n:127.0.0.1:50267_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=20
[junit4] 2> 808796 DEBUG (zkCallback-1603-thread-2-processing-n:127.0.0.1:63652_solr) [n:127.0.0.1:63652_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 808797 DEBUG (zkCallback-1603-thread-2-processing-n:127.0.0.1:63652_solr) [n:127.0.0.1:63652_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:50267_solr, 127.0.0.1:63987_solr, 127.0.0.1:63652_solr]
[junit4] 2> 808798 DEBUG (OverseerAutoScalingTriggerThread-72134384416718853-127.0.0.1:63652_solr-n_0000000000) [n:127.0.0.1:63652_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 808800 INFO (OverseerAutoScalingTriggerThread-72134384416718853-127.0.0.1:63652_solr-n_0000000000) [n:127.0.0.1:63652_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 808801 DEBUG (OverseerAutoScalingTriggerThread-72134384416718853-127.0.0.1:63652_solr-n_0000000000) [n:127.0.0.1:63652_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 808802 DEBUG (OverseerAutoScalingTriggerThread-72134384416718853-127.0.0.1:63652_solr-n_0000000000) [n:127.0.0.1:63652_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 808803 DEBUG (ScheduledTrigger-2380-thread-1) [n:127.0.0.1:63652_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger with currently live nodes: 3
[junit4] 2> 808806 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[21E318AF226E3481]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@3a98f52e{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 808806 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[21E318AF226E3481]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=687298801
[junit4] 2> 808806 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[21E318AF226E3481]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 808806 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[21E318AF226E3481]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@50edaa2d: rootName = solr_63987, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@3377bc3
[junit4] 2> 808814 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[21E318AF226E3481]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 808814 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[21E318AF226E3481]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6b99f17b: rootName = solr_63987, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@328056fc
[junit4] 2> 808820 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[21E318AF226E3481]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 808820 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[21E318AF226E3481]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@36bdd092: rootName = solr_63987, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@1a3a3fd9
[junit4] 2> 808824 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[21E318AF226E3481]) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:63987_solr
[junit4] 2> 808826 INFO (zkCallback-1623-thread-1-processing-n:127.0.0.1:63987_solr) [n:127.0.0.1:63987_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 808826 INFO (zkCallback-1603-thread-2-processing-n:127.0.0.1:63652_solr) [n:127.0.0.1:63652_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 808827 INFO (zkCallback-1605-thread-1-processing-n:127.0.0.1:50267_solr) [n:127.0.0.1:50267_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 808827 INFO (zkCallback-1615-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 808829 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[21E318AF226E3481]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 808830 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[21E318AF226E3481]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 808834 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[21E318AF226E3481]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@719f430b{/solr,null,UNAVAILABLE}
[junit4] 2> 808834 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[21E318AF226E3481]) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 809806 DEBUG (ScheduledTrigger-2380-thread-2) [n:127.0.0.1:63652_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger with currently live nodes: 2
[junit4] 2> 809807 DEBUG (ScheduledTrigger-2380-thread-2) [n:127.0.0.1:63652_solr ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:63987_solr
[junit4] 2> 810339 INFO (qtp1300884583-7326) [n:127.0.0.1:50267_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 810342 INFO (qtp1072870045-7312) [n:127.0.0.1:63652_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=1
[junit4] 2> 810342 DEBUG (qtp1072870045-7311) [n:127.0.0.1:63652_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 810344 INFO (qtp1072870045-7311) [n:127.0.0.1:63652_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=6
[junit4] 2> 810345 DEBUG (zkCallback-1603-thread-2-processing-n:127.0.0.1:63652_solr) [n:127.0.0.1:63652_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 810346 DEBUG (zkCallback-1603-thread-2-processing-n:127.0.0.1:63652_solr) [n:127.0.0.1:63652_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:50267_solr, 127.0.0.1:63652_solr]
[junit4] 2> 810346 DEBUG (OverseerAutoScalingTriggerThread-72134384416718853-127.0.0.1:63652_solr-n_0000000000) [n:127.0.0.1:63652_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 810346 INFO (OverseerAutoScalingTriggerThread-72134384416718853-127.0.0.1:63652_solr-n_0000000000) [n:127.0.0.1:63652_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 810347 DEBUG (OverseerAutoScalingTriggerThread-72134384416718853-127.0.0.1:63652_solr-n_0000000000) [n:127.0.0.1:63652_solr ] o.a.s.c.a.NodeLostTrigger Adding lost node from marker path: 127.0.0.1:63987_solr
[junit4] 2> 810347 DEBUG (OverseerAutoScalingTriggerThread-72134384416718853-127.0.0.1:63652_solr-n_0000000000) [n:127.0.0.1:63652_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 810347 DEBUG (ScheduledTrigger-2380-thread-1) [n:127.0.0.1:63652_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger with currently live nodes: 2
[junit4] 2> 810347 DEBUG (ScheduledTrigger-2380-thread-1) [n:127.0.0.1:63652_solr ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:63987_solr]
[junit4] 2> 810348 DEBUG (ScheduledTrigger-2380-thread-1) [n:127.0.0.1:63652_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"429aecb163511T7dgy1hlc7wmzgmuwn6xdamlzr",
[junit4] 2> "source":"node_lost_restore_trigger",
[junit4] 2> "eventTime":1171730615121169,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1171730615121169],
[junit4] 2> "nodeNames":["127.0.0.1:63987_solr"]}}
[junit4] 2> 810348 DEBUG (OverseerAutoScalingTriggerThread-72134384416718853-127.0.0.1:63652_solr-n_0000000000) [n:127.0.0.1:63652_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 810348 DEBUG (ScheduledTrigger-2380-thread-1) [n:127.0.0.1:63652_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_lost_restore_trigger]
[junit4] 2> 810349 INFO (ScheduledTrigger-2380-thread-1) [n:127.0.0.1:63652_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 810350 DEBUG (AutoscalingActionExecutor-2381-thread-1-processing-n:127.0.0.1:63652_solr) [n:127.0.0.1:63652_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"429aecb163511T7dgy1hlc7wmzgmuwn6xdamlzr",
[junit4] 2> "source":"node_lost_restore_trigger",
[junit4] 2> "eventTime":1171730615121169,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1171730615121169],
[junit4] 2> "_enqueue_time_":1171730616802805,
[junit4] 2> "nodeNames":["127.0.0.1:63987_solr"]}}
[junit4] 2> 810350 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[21E318AF226E3481]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testNodeLostTriggerRestoreState
[junit4] 2> 810354 DEBUG (AutoscalingActionExecutor-2381-thread-1-processing-n:127.0.0.1:63652_solr) [n:127.0.0.1:63652_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_restore_trigger after 5000ms
[junit4] 2> 810354 DEBUG (AutoscalingActionExecutor-2381-thread-1-processing-n:127.0.0.1:63652_solr) [n:127.0.0.1:63652_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 4 ms for event id=429aecb163511T7dgy1hlc7wmzgmuwn6xdamlzr
[junit4] 2> 810366 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testCooldown
[junit4] 2> 810367 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 5
[junit4] 2> 810367 DEBUG (zkCallback-1603-thread-2-processing-n:127.0.0.1:63652_solr) [n:127.0.0.1:63652_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
[junit4] 2> 810367 DEBUG (OverseerAutoScalingTriggerThread-72134384416718853-127.0.0.1:63652_solr-n_0000000000) [n:127.0.0.1:63652_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
[junit4] 2> 810369 DEBUG (OverseerAutoScalingTriggerThread-72134384416718853-127.0.0.1:63652_solr-n_0000000000) [n:127.0.0.1:63652_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 810369 DEBUG (OverseerAutoScalingTriggerThread-72134384416718853-127.0.0.1:63652_solr-n_0000000000) [n:127.0.0.1:63652_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 810370 DEBUG (OverseerAutoScalingTriggerThread-72134384416718853-127.0.0.1:63652_solr-n_0000000000) [n:127.0.0.1:63652_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
[junit4] 2> 810372 INFO (qtp1300884583-7323) [n:127.0.0.1:50267_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 810380 INFO (qtp1072870045-7312) [n:127.0.0.1:63652_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 810381 DEBUG (qtp1072870045-7315) [n:127.0.0.1:63652_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 810381 INFO (qtp1072870045-7315) [n:127.0.0.1:63652_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=11
[junit4] 2> 810383 DEBUG (zkCallback-1603-thread-2-processing-n:127.0.0.1:63652_solr) [n:127.0.0.1:63652_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
[junit4] 2> 810383 DEBUG (zkCallback-1603-thread-2-processing-n:127.0.0.1:63652_solr) [n:127.0.0.1:63652_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:50267_solr, 127.0.0.1:63652_solr]
[junit4] 2> 810384 DEBUG (zkCallback-1603-thread-2-processing-n:127.0.0.1:63652_solr) [n:127.0.0.1:63652_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestTriggerAction}], enabled=true}
[junit4] 2> 810385 DEBUG (OverseerAutoScalingTriggerThread-72134384416718853-127.0.0.1:63652_solr-n_0000000000) [n:127.0.0.1:63652_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
[junit4] 2> 810385 INFO (qtp1300884583-7323) [n:127.0.0.1:50267_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 810387 INFO (qtp1072870045-7319) [n:127.0.0.1:63652_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 810387 INFO (OverseerAutoScalingTriggerThread-72134384416718853-127.0.0.1:63652_solr-n_0000000000) [n:127.0.0.1:63652_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 810388 DEBUG (OverseerAutoScalingTriggerThread-72134384416718853-127.0.0.1:63652_solr-n_0000000000) [n:127.0.0.1:63652_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 810388 DEBUG (OverseerAutoScalingTriggerThread-72134384416718853-127.0.0.1:63652_solr-n_0000000000) [n:127.0.0.1:63652_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
[junit4] 2> 810388 DEBUG (ScheduledTrigger-2380-thread-2) [n:127.0.0.1:63652_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
[junit4] 2> 810388 DEBUG (ScheduledTrigger-2380-thread-2) [n:127.0.0.1:63652_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 810388 DEBUG (qtp1300884583-7326) [n:127.0.0.1:50267_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 810389 INFO (qtp1300884583-7326) [n:127.0.0.1:50267_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=6
[junit4] 2> 810390 DEBUG (zkCallback-1603-thread-2-processing-n:127.0.0.1:63652_solr) [n:127.0.0.1:63652_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
[junit4] 2> 810391 DEBUG (zkCallback-1603-thread-2-processing-n:127.0.0.1:63652_solr) [n:127.0.0.1:63652_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:50267_solr, 127.0.0.1:63652_solr]
[junit4] 2> 810391 DEBUG (zkCallback-1603-thread-2-processing-n:127.0.0.1:63652_solr) [n:127.0.0.1:63652_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestTriggerAction}], enabled=true}
[junit4] 2> 810391 DEBUG (OverseerAutoScalingTriggerThread-72134384416718853-127.0.0.1:63652_solr-n_0000000000) [n:127.0.0.1:63652_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
[junit4] 2> 810391 DEBUG (OverseerAutoScalingTriggerThread-72134384416718853-127.0.0.1:63652_solr-n_0000000000) [n:127.0.0.1:63652_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 810392 DEBUG (OverseerAutoScalingTriggerThread-72134384416718853-127.0.0.1:63652_solr-n_0000000000) [n:127.0.0.1:63652_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
[junit4] 2> 810393 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T20:27:37-01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 810393 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 810393 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 810393 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 810394 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6dc2cbc9{/solr,null,AVAILABLE}
[junit4] 2> 810394 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7b745a0a{HTTP/1.1,[http/1.1]}{127.0.0.1:52204}
[junit4] 2> 810394 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.e.j.s.Server Started @813199ms
[junit4] 2> 810394 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=52204}
[junit4] 2> 810394 ERROR (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 810399 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 810399 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 810399 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 810399 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-03T11:12:42.911Z
[junit4] 2> 810418 INFO (zkConnectionManagerCallback-1628-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 810422 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 810432 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41487/solr
[junit4] 2> 810436 INFO (zkConnectionManagerCallback-1632-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 810440 INFO (zkConnectionManagerCallback-1634-thread-1-processing-n:127.0.0.1:52204_solr) [n:127.0.0.1:52204_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 810447 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [n:127.0.0.1:52204_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 810449 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [n:127.0.0.1:52204_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 810451 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [n:127.0.0.1:52204_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52204_solr
[junit4] 2> 810454 INFO (zkCallback-1603-thread-2-processing-n:127.0.0.1:63652_solr) [n:127.0.0.1:63652_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 810454 INFO (zkCallback-1615-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 810454 INFO (zkCallback-1605-thread-1-processing-n:127.0.0.1:50267_solr) [n:127.0.0.1:50267_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 810455 INFO (zkCallback-1633-thread-1-processing-n:127.0.0.1:52204_solr) [n:127.0.0.1:52204_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 810775 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [n:127.0.0.1:52204_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_52204.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67aa6241
[junit4] 2> 810782 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [n:127.0.0.1:52204_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_52204.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67aa6241
[junit4] 2> 810782 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [n:127.0.0.1:52204_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_52204.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67aa6241
[junit4] 2> 810783 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [n:127.0.0.1:52204_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_21E318AF226E3481-001/tempDir-001/node4/.
[junit4] 2> 811391 DEBUG (ScheduledTrigger-2380-thread-2) [n:127.0.0.1:63652_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
[junit4] 2> 811391 DEBUG (ScheduledTrigger-2380-thread-2) [n:127.0.0.1:63652_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 811392 DEBUG (ScheduledTrigger-2380-thread-2) [n:127.0.0.1:63652_solr ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:52204_solr at time 1171731660392650
[junit4] 2> 812394 DEBUG (ScheduledTrigger-2380-thread-2) [n:127.0.0.1:63652_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
[junit4] 2> 812394 DEBUG (ScheduledTrigger-2380-thread-2) [n:127.0.0.1:63652_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 812394 DEBUG (ScheduledTrigger-2380-thread-2) [n:127.0.0.1:63652_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger firing registered processor for nodes: [127.0.0.1:52204_solr] added at times [1171731660392650], now=1171732663136723
[junit4] 2> 812394 DEBUG (ScheduledTrigger-2380-thread-2) [n:127.0.0.1:63652_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"429af0963c8caT7dgy1hlc7wmzgmuwn6xdamlzt",
[junit4] 2> "source":"node_added_cooldown_trigger",
[junit4] 2> "eventTime":1171731660392650,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1171731660392650],
[junit4] 2> "nodeNames":["127.0.0.1:52204_solr"]}}
[junit4] 2> 812394 DEBUG (ScheduledTrigger-2380-thread-2) [n:127.0.0.1:63652_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_added_cooldown_trigger]
[junit4] 2> 812395 INFO (ScheduledTrigger-2380-thread-2) [n:127.0.0.1:63652_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 812395 DEBUG (AutoscalingActionExecutor-2381-thread-1-processing-n:127.0.0.1:63652_solr) [n:127.0.0.1:63652_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"429af0963c8caT7dgy1hlc7wmzgmuwn6xdamlzt",
[junit4] 2> "source":"node_added_cooldown_trigger",
[junit4] 2> "eventTime":1171731660392650,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1171731660392650],
[junit4] 2> "_enqueue_time_":1171732663293422,
[junit4] 2> "nodeNames":["127.0.0.1:52204_solr"]}}
[junit4] 2> 812399 DEBUG (AutoscalingActionExecutor-2381-thread-1-processing-n:127.0.0.1:63652_solr) [n:127.0.0.1:63652_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_cooldown_trigger after 5000ms
[junit4] 2> 812399 DEBUG (AutoscalingActionExecutor-2381-thread-1-processing-n:127.0.0.1:63652_solr) [n:127.0.0.1:63652_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 3 ms for event id=429af0963c8caT7dgy1hlc7wmzgmuwn6xdamlzt
[junit4] 2> 813401 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T20:27:37-01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 813402 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 813402 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 813402 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 813402 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@29747017{/solr,null,AVAILABLE}
[junit4] 2> 813403 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@13596553{HTTP/1.1,[http/1.1]}{127.0.0.1:47487}
[junit4] 2> 813403 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.e.j.s.Server Started @816209ms
[junit4] 2> 813403 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=47487}
[junit4] 2> 813403 ERROR (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 813404 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 813404 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 813404 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 813404 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-03T11:12:45.916Z
[junit4] 2> 813406 INFO (zkConnectionManagerCallback-1638-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 813408 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 813417 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41487/solr
[junit4] 2> 813419 INFO (zkConnectionManagerCallback-1642-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 813423 INFO (zkConnectionManagerCallback-1644-thread-1-processing-n:127.0.0.1:47487_solr) [n:127.0.0.1:47487_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 813434 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [n:127.0.0.1:47487_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 813438 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [n:127.0.0.1:47487_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 813442 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [n:127.0.0.1:47487_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47487_solr
[junit4] 2> 813444 INFO (zkCallback-1603-thread-2-processing-n:127.0.0.1:63652_solr) [n:127.0.0.1:63652_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 813444 INFO (zkCallback-1633-thread-1-processing-n:127.0.0.1:52204_solr) [n:127.0.0.1:52204_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 813444 INFO (zkCallback-1605-thread-1-processing-n:127.0.0.1:50267_solr) [n:127.0.0.1:50267_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 813444 INFO (zkCallback-1615-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 813445 INFO (zkCallback-1643-thread-1-processing-n:127.0.0.1:47487_solr) [n:127.0.0.1:47487_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 813629 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [n:127.0.0.1:47487_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_47487.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67aa6241
[junit4] 2> 813636 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [n:127.0.0.1:47487_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_47487.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67aa6241
[junit4] 2> 813636 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [n:127.0.0.1:47487_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_47487.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67aa6241
[junit4] 2> 813637 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [n:127.0.0.1:47487_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_21E318AF226E3481-001/tempDir-001/node5/.
[junit4] 2> 817400 DEBUG (ScheduledTrigger-2380-thread-1) [n:127.0.0.1:63652_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
[junit4] 2> 817400 DEBUG (ScheduledTrigger-2380-thread-1) [n:127.0.0.1:63652_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 817400 DEBUG (ScheduledTrigger-2380-thread-1) [n:127.0.0.1:63652_solr ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:47487_solr at time 1171737671166870
[junit4] 2> 818410 DEBUG (ScheduledTrigger-2380-thread-1) [n:127.0.0.1:63652_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_cooldown_trigger
[junit4] 2> 818410 DEBUG (ScheduledTrigger-2380-thread-1) [n:127.0.0.1:63652_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 818410 DEBUG (ScheduledTrigger-2380-thread-1) [n:127.0.0.1:63652_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger firing registered processor for nodes: [127.0.0.1:47487_solr] added at times [1171737671166870], now=1171738681071969
[junit4] 2> 818410 DEBUG (ScheduledTrigger-2380-thread-1) [n:127.0.0.1:63652_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"429b06fa8eb96T7dgy1hlc7wmzgmuwn6xdamlzv",
[junit4] 2> "source":"node_added_cooldown_trigger",
[junit4] 2> "eventTime":1171737671166870,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1171737671166870],
[junit4] 2> "nodeNames":["127.0.0.1:47487_solr"]}}
[junit4] 2> 818410 DEBUG (ScheduledTrigger-2380-thread-1) [n:127.0.0.1:63652_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_added_cooldown_trigger]
[junit4] 2> 818417 DEBUG (AutoscalingActionExecutor-2381-thread-1-processing-n:127.0.0.1:63652_solr) [n:127.0.0.1:63652_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"429b06fa8eb96T7dgy1hlc7wmzgmuwn6xdamlzv",
[junit4] 2> "source":"node_added_cooldown_trigger",
[junit4] 2> "eventTime":1171737671166870,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1171737671166870],
[junit4] 2> "_enqueue_time_":1171738681245347,
[junit4] 2> "nodeNames":["127.0.0.1:47487_solr"]}}
[junit4] 2> 818418 DEBUG (AutoscalingActionExecutor-2381-thread-1-processing-n:127.0.0.1:63652_solr) [n:127.0.0.1:63652_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_cooldown_trigger after 5000ms
[junit4] 2> 818418 DEBUG (AutoscalingActionExecutor-2381-thread-1-processing-n:127.0.0.1:63652_solr) [n:127.0.0.1:63652_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 1 ms for event id=429b06fa8eb96T7dgy1hlc7wmzgmuwn6xdamlzv
[junit4] 2> 820429 INFO (qtp147815030-7427) [n:127.0.0.1:47487_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=1
[junit4] 2> 820432 INFO (qtp1300884583-7324) [n:127.0.0.1:50267_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 820435 INFO (qtp1305184980-7403) [n:127.0.0.1:52204_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=1
[junit4] 2> 820437 INFO (qtp1072870045-7319) [n:127.0.0.1:63652_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 820439 DEBUG (qtp1072870045-7315) [n:127.0.0.1:63652_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 820440 INFO (qtp1072870045-7315) [n:127.0.0.1:63652_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=21
[junit4] 2> 820441 DEBUG (zkCallback-1603-thread-2-processing-n:127.0.0.1:63652_solr) [n:127.0.0.1:63652_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 8
[junit4] 2> 820442 DEBUG (zkCallback-1603-thread-2-processing-n:127.0.0.1:63652_solr) [n:127.0.0.1:63652_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:50267_solr, 127.0.0.1:47487_solr, 127.0.0.1:52204_solr, 127.0.0.1:63652_solr]
[junit4] 2> 820442 DEBUG (zkCallback-1603-thread-2-processing-n:127.0.0.1:63652_solr) [n:127.0.0.1:63652_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_cooldown_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestTriggerAction}], enabled=true}
[junit4] 2> 820442 DEBUG (OverseerAutoScalingTriggerThread-72134384416718853-127.0.0.1:63652_solr-n_0000000000) [n:127.0.0.1:63652_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
[junit4] 2> 820442 DEBUG (OverseerAutoScalingTriggerThread-72134384416718853-127.0.0.1:63652_solr-n_0000000000) [n:127.0.0.1:63652_solr ] o.a.s.c.a.ScheduledTriggers Changing value of autoscaling property: triggerCooldownPeriodSeconds from: 5 to: 7
[junit4] 2> 820443 DEBUG (OverseerAutoScalingTriggerThread-72134384416718853-127.0.0.1:63652_solr-n_0000000000) [n:127.0.0.1:63652_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 820443 DEBUG (OverseerAutoScalingTriggerThread-72134384416718853-127.0.0.1:63652_solr-n_0000000000) [n:127.0.0.1:63652_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion 8
[junit4] 2> 820451 INFO (qtp147815030-7429) [n:127.0.0.1:47487_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=6
[junit4] 2> 820474 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T20:27:37-01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 820475 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 820475 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 820475 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 820475 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4c12df4{/solr,null,AVAILABLE}
[junit4] 2> 820475 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@b05e539{HTTP/1.1,[http/1.1]}{127.0.0.1:47025}
[junit4] 2> 820475 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.e.j.s.Server Started @823283ms
[junit4] 2> 820475 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=47025}
[junit4] 2> 820476 ERROR (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 820476 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 820476 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 820476 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 820476 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-03T11:12:52.988Z
[junit4] 2> 820480 INFO (zkConnectionManagerCallback-1648-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 820481 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 820491 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41487/solr
[junit4] 2> 820493 INFO (zkConnectionManagerCallback-1652-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 820497 INFO (zkConnectionManagerCallback-1654-thread-1-processing-n:127.0.0.1:47025_solr) [n:127.0.0.1:47025_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 820505 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [n:127.0.0.1:47025_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 820508 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [n:127.0.0.1:47025_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 820511 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [n:127.0.0.1:47025_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47025_solr
[junit4] 2> 820513 INFO (zkCallback-1643-thread-1-processing-n:127.0.0.1:47487_solr) [n:127.0.0.1:47487_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 820513 INFO (zkCallback-1603-thread-2-processing-n:127.0.0.1:63652_solr) [n:127.0.0.1:63652_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 820515 INFO (zkCallback-1653-thread-1-processing-n:127.0.0.1:47025_solr) [n:127.0.0.1:47025_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 820515 INFO (zkCallback-1633-thread-1-processing-n:127.0.0.1:52204_solr) [n:127.0.0.1:52204_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 820516 INFO (zkCallback-1615-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 820516 INFO (zkCallback-1605-thread-1-processing-n:127.0.0.1:50267_solr) [n:127.0.0.1:50267_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 820857 INFO (TEST-TriggerIntegrationTest.testCooldown-seed#[21E318AF226E3481]) [n:127.0.0.1:47025_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_47025.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@67aa6241
[junit4] 2> 820890 INFO (TEST-TriggerIntegrationTest.
[...truncated too long message...]
6DD7D06A7]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 12156 INFO (coreCloseExecutor-75-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@2fd01fea
[junit4] 2> 12156 INFO (coreCloseExecutor-75-thread-1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=802168810
[junit4] 2> 12169 INFO (SUITE-TestFieldLengthFeature-seed#[AF5A6D86DD7D06A7]-worker) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@7b81570a{/solr,null,UNAVAILABLE}
[junit4] 2> 12169 INFO (SUITE-TestFieldLengthFeature-seed#[AF5A6D86DD7D06A7]-worker) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 42168 WARN (SUITE-TestFieldLengthFeature-seed#[AF5A6D86DD7D06A7]-worker) [ ] o.e.j.u.t.QueuedThreadPool QueuedThreadPool@qtp1643342153{STOPPING,8<=8<=10000,i=0,q=1} Couldn't stop Thread[qtp1643342153-86,5,TGRP-TestFieldLengthFeature]
[junit4] 2> 42170 INFO (SUITE-TestFieldLengthFeature-seed#[AF5A6D86DD7D06A7]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> 42170 INFO (SUITE-TestFieldLengthFeature-seed#[AF5A6D86DD7D06A7]-worker) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=777581082
[junit4] 2> 42170 INFO (SUITE-TestFieldLengthFeature-seed#[AF5A6D86DD7D06A7]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 42170 INFO (SUITE-TestFieldLengthFeature-seed#[AF5A6D86DD7D06A7]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@25514ff5: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / null
[junit4] 2> 42170 INFO (SUITE-TestFieldLengthFeature-seed#[AF5A6D86DD7D06A7]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 42170 INFO (SUITE-TestFieldLengthFeature-seed#[AF5A6D86DD7D06A7]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@48a1ca93: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / null
[junit4] 2> 42170 INFO (SUITE-TestFieldLengthFeature-seed#[AF5A6D86DD7D06A7]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 42170 INFO (SUITE-TestFieldLengthFeature-seed#[AF5A6D86DD7D06A7]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3d9be872: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / null
[junit4] 2> 42172 INFO (coreCloseExecutor-77-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@24d9a789
[junit4] 2> 42172 INFO (coreCloseExecutor-77-thread-1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=618243977
[junit4] 2> 42172 INFO (coreCloseExecutor-77-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@28199d87: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / null
[junit4] 2> 42176 INFO (SUITE-TestFieldLengthFeature-seed#[AF5A6D86DD7D06A7]-worker) [ ] o.a.s.SolrTestCaseJ4 ------------------------------------------------------- Done waiting for tracked resources to be released
[junit4] 2> Feb 03, 2018 12:22:03 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> Feb 03, 2018 12:22:23 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.solr.ltr.feature.TestFieldLengthFeature:
[junit4] 2> 1) Thread[id=86, name=qtp1643342153-86, state=TIMED_WAITING, group=TGRP-TestFieldLengthFeature]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Feb 03, 2018 12:22:23 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=86, name=qtp1643342153-86, state=TIMED_WAITING, group=TGRP-TestFieldLengthFeature]
[junit4] 2> Feb 03, 2018 12:22:26 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> SEVERE: There are still zombie threads that couldn't be terminated:
[junit4] 2> 1) Thread[id=86, name=qtp1643342153-86, state=TIMED_WAITING, group=TGRP-TestFieldLengthFeature]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {description=FST50, id=PostingsFormat(name=Direct), text=PostingsFormat(name=Memory), title=PostingsFormat(name=Asserting)}, docValues:{_version_=DocValuesFormat(name=Lucene70)}, maxPointsInLeafNode=561, maxMBSortInHeap=7.412277276344565, sim=RandomSimilarity(queryNorm=false): {}, locale=ar-SA, timezone=EST5EDT
[junit4] 2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=3,threads=2,free=33575832,total=100663296
[junit4] 2> NOTE: All tests run in this JVM: [TestUserTermScorerQuery, TestLinearModel, TestFieldLengthFeature]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestFieldLengthFeature -Dtests.seed=AF5A6D86DD7D06A7 -Dtests.slow=true -Dtests.locale=ar-SA -Dtests.timezone=EST5EDT -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J1 | TestFieldLengthFeature (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.ltr.feature.TestFieldLengthFeature:
[junit4] > 1) Thread[id=86, name=qtp1643342153-86, state=TIMED_WAITING, group=TGRP-TestFieldLengthFeature]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([AF5A6D86DD7D06A7]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=86, name=qtp1643342153-86, state=TIMED_WAITING, group=TGRP-TestFieldLengthFeature]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([AF5A6D86DD7D06A7]:0)
[junit4] Completed [29/35 (1!)] on J1 in 55.16s, 3 tests, 2 errors <<< FAILURES!
[...truncated 40809 lines...]
[JENKINS] Lucene-Solr-7.x-Solaris (64bit/jdk1.8.0) - Build # 426 -
Still Unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Solaris/426/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC
6 tests failed.
FAILED: org.apache.solr.cloud.MoveReplicaHDFSTest.testNormalFailedMove
Error Message:
Collection not found: MoveReplicaHDFSTest_failed_coll_false
Stack Trace:
org.apache.solr.common.SolrException: Collection not found: MoveReplicaHDFSTest_failed_coll_false
at __randomizedtesting.SeedInfo.seed([48EB1A90529CBD0F:6FE005D87EDE2316]:0)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:851)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:173)
at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:138)
at org.apache.solr.cloud.MoveReplicaTest.addDocs(MoveReplicaTest.java:390)
at org.apache.solr.cloud.MoveReplicaTest.testFailedMove(MoveReplicaTest.java:263)
at org.apache.solr.cloud.MoveReplicaHDFSTest.testNormalFailedMove(MoveReplicaHDFSTest.java:67)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testNodeLostTrigger
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([48EB1A90529CBD0F:4C15B38268563F95]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testNodeLostTrigger(TriggerIntegrationTest.java:521)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.client.solrj.request.SchemaTest
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.client.solrj.request.SchemaTest: 1) Thread[id=3235, name=qtp1073869087-3235, state=TIMED_WAITING, group=TGRP-SchemaTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.client.solrj.request.SchemaTest:
1) Thread[id=3235, name=qtp1073869087-3235, state=TIMED_WAITING, group=TGRP-SchemaTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([2B21A3CEC8AAF2DD]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.client.solrj.request.SchemaTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=3235, name=qtp1073869087-3235, state=TIMED_WAITING, group=TGRP-SchemaTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=3235, name=qtp1073869087-3235, state=TIMED_WAITING, group=TGRP-SchemaTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([2B21A3CEC8AAF2DD]:0)
FAILED: org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitWithChaosMonkey
Error Message:
There are still nodes recoverying - waited for 330 seconds
Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 330 seconds
at __randomizedtesting.SeedInfo.seed([48EB1A90529CBD0F:C3CCC941139A168B]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:185)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:140)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:135)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:915)
at org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitWithChaosMonkey(ShardSplitTest.java:431)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testMetricTrigger
Error Message:
The trigger did not fire at all
Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at __randomizedtesting.SeedInfo.seed([48EB1A90529CBD0F:F2E72D1F0D746B40]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testMetricTrigger(TriggerIntegrationTest.java:1608)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12149 lines...]
[junit4] Suite: org.apache.solr.cloud.MoveReplicaHDFSTest
[junit4] 2> 852839 INFO (SUITE-MoveReplicaHDFSTest-seed#[48EB1A90529CBD0F]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_48EB1A90529CBD0F-001/init-core-data-001
[junit4] 2> 852840 INFO (SUITE-MoveReplicaHDFSTest-seed#[48EB1A90529CBD0F]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 852841 INFO (SUITE-MoveReplicaHDFSTest-seed#[48EB1A90529CBD0F]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 852841 INFO (SUITE-MoveReplicaHDFSTest-seed#[48EB1A90529CBD0F]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_48EB1A90529CBD0F-001/tempDir-001
[junit4] 2> 852841 INFO (SUITE-MoveReplicaHDFSTest-seed#[48EB1A90529CBD0F]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 852841 INFO (Thread-2051) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 852841 INFO (Thread-2051) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 852851 ERROR (Thread-2051) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 852942 INFO (SUITE-MoveReplicaHDFSTest-seed#[48EB1A90529CBD0F]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:34090
[junit4] 2> 852945 INFO (zkConnectionManagerCallback-1419-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 852953 INFO (jetty-launcher-1416-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T22:27:37+01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 852953 INFO (jetty-launcher-1416-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T22:27:37+01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 852953 INFO (jetty-launcher-1416-thread-3) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T22:27:37+01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 852953 INFO (jetty-launcher-1416-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 852953 INFO (jetty-launcher-1416-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 852953 INFO (jetty-launcher-1416-thread-4) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T22:27:37+01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 852953 INFO (jetty-launcher-1416-thread-2) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 852954 INFO (jetty-launcher-1416-thread-4) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 852954 INFO (jetty-launcher-1416-thread-4) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 852954 INFO (jetty-launcher-1416-thread-4) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 852955 INFO (jetty-launcher-1416-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 852955 INFO (jetty-launcher-1416-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 852955 INFO (jetty-launcher-1416-thread-1) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 852956 INFO (jetty-launcher-1416-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6bda347d{/solr,null,AVAILABLE}
[junit4] 2> 852956 INFO (jetty-launcher-1416-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@676f3318{HTTP/1.1,[http/1.1]}{127.0.0.1:46272}
[junit4] 2> 852956 INFO (jetty-launcher-1416-thread-1) [ ] o.e.j.s.Server Started @856475ms
[junit4] 2> 852956 INFO (jetty-launcher-1416-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=46272}
[junit4] 2> 852956 ERROR (jetty-launcher-1416-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 852957 INFO (jetty-launcher-1416-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 852957 INFO (jetty-launcher-1416-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 852957 INFO (jetty-launcher-1416-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 852957 INFO (jetty-launcher-1416-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-03T22:15:50.645Z
[junit4] 2> 852959 INFO (jetty-launcher-1416-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@57c08bcc{/solr,null,AVAILABLE}
[junit4] 2> 852960 INFO (jetty-launcher-1416-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3b29f130{/solr,null,AVAILABLE}
[junit4] 2> 852964 INFO (jetty-launcher-1416-thread-4) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2b924f9d{HTTP/1.1,[http/1.1]}{127.0.0.1:38378}
[junit4] 2> 852964 INFO (jetty-launcher-1416-thread-3) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 852964 INFO (jetty-launcher-1416-thread-3) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 852964 INFO (jetty-launcher-1416-thread-3) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 852965 INFO (jetty-launcher-1416-thread-4) [ ] o.e.j.s.Server Started @856483ms
[junit4] 2> 852965 INFO (jetty-launcher-1416-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=38378}
[junit4] 2> 852965 ERROR (jetty-launcher-1416-thread-4) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 852964 INFO (jetty-launcher-1416-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1f22d5e0{HTTP/1.1,[http/1.1]}{127.0.0.1:64028}
[junit4] 2> 852965 INFO (jetty-launcher-1416-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 852965 INFO (jetty-launcher-1416-thread-2) [ ] o.e.j.s.Server Started @856484ms
[junit4] 2> 852965 INFO (jetty-launcher-1416-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=64028}
[junit4] 2> 852965 INFO (jetty-launcher-1416-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 852965 INFO (jetty-launcher-1416-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 852965 INFO (jetty-launcher-1416-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-03T22:15:50.653Z
[junit4] 2> 852965 ERROR (jetty-launcher-1416-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 852965 INFO (jetty-launcher-1416-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@27769fc1{/solr,null,AVAILABLE}
[junit4] 2> 852965 INFO (jetty-launcher-1416-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 852965 INFO (jetty-launcher-1416-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 852965 INFO (jetty-launcher-1416-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 852965 INFO (jetty-launcher-1416-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-03T22:15:50.653Z
[junit4] 2> 852966 INFO (jetty-launcher-1416-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1b3edb80{HTTP/1.1,[http/1.1]}{127.0.0.1:35438}
[junit4] 2> 852966 INFO (jetty-launcher-1416-thread-3) [ ] o.e.j.s.Server Started @856484ms
[junit4] 2> 852966 INFO (jetty-launcher-1416-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35438}
[junit4] 2> 852966 ERROR (jetty-launcher-1416-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 852966 INFO (jetty-launcher-1416-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 852966 INFO (jetty-launcher-1416-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 852966 INFO (jetty-launcher-1416-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 852966 INFO (jetty-launcher-1416-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-03T22:15:50.654Z
[junit4] 2> 852969 INFO (zkConnectionManagerCallback-1423-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 852969 INFO (zkConnectionManagerCallback-1427-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 852970 INFO (jetty-launcher-1416-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 852978 INFO (zkConnectionManagerCallback-1421-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 852979 INFO (zkConnectionManagerCallback-1425-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 852979 INFO (jetty-launcher-1416-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 852979 INFO (jetty-launcher-1416-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 852980 INFO (jetty-launcher-1416-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34090/solr
[junit4] 2> 852981 INFO (jetty-launcher-1416-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 852983 INFO (zkConnectionManagerCallback-1431-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 852994 INFO (zkConnectionManagerCallback-1433-thread-1-processing-n:127.0.0.1:35438_solr) [n:127.0.0.1:35438_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 852998 INFO (jetty-launcher-1416-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34090/solr
[junit4] 2> 853001 INFO (jetty-launcher-1416-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34090/solr
[junit4] 2> 853001 INFO (jetty-launcher-1416-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34090/solr
[junit4] 2> 853003 INFO (zkConnectionManagerCallback-1437-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 853005 INFO (zkConnectionManagerCallback-1443-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 853005 INFO (zkConnectionManagerCallback-1445-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 853006 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100483717410007, likely client has closed socket
[junit4] 2> 853006 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100483717410008, likely client has closed socket
[junit4] 2> 853012 INFO (zkConnectionManagerCallback-1447-thread-1-processing-n:127.0.0.1:64028_solr) [n:127.0.0.1:64028_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 853019 INFO (zkConnectionManagerCallback-1449-thread-1-processing-n:127.0.0.1:46272_solr) [n:127.0.0.1:46272_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 853020 INFO (zkConnectionManagerCallback-1451-thread-1-processing-n:127.0.0.1:38378_solr) [n:127.0.0.1:38378_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 853052 INFO (jetty-launcher-1416-thread-4) [n:127.0.0.1:38378_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 853052 INFO (jetty-launcher-1416-thread-1) [n:127.0.0.1:46272_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 853052 INFO (jetty-launcher-1416-thread-4) [n:127.0.0.1:38378_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:38378_solr
[junit4] 2> 853054 INFO (jetty-launcher-1416-thread-4) [n:127.0.0.1:38378_solr ] o.a.s.c.Overseer Overseer (id=72136995488464908-127.0.0.1:38378_solr-n_0000000000) starting
[junit4] 2> 853059 INFO (jetty-launcher-1416-thread-1) [n:127.0.0.1:46272_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46272_solr
[junit4] 2> 853060 INFO (jetty-launcher-1416-thread-3) [n:127.0.0.1:35438_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 853063 INFO (zkCallback-1432-thread-1-processing-n:127.0.0.1:35438_solr) [n:127.0.0.1:35438_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 853064 INFO (zkCallback-1448-thread-1-processing-n:127.0.0.1:46272_solr) [n:127.0.0.1:46272_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 853065 INFO (jetty-launcher-1416-thread-3) [n:127.0.0.1:35438_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35438_solr
[junit4] 2> 853066 INFO (zkCallback-1448-thread-1-processing-n:127.0.0.1:46272_solr) [n:127.0.0.1:46272_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 853114 INFO (zkCallback-1450-thread-1-processing-n:127.0.0.1:38378_solr) [n:127.0.0.1:38378_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 853115 INFO (zkCallback-1432-thread-1-processing-n:127.0.0.1:35438_solr) [n:127.0.0.1:35438_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 853122 INFO (jetty-launcher-1416-thread-2) [n:127.0.0.1:64028_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 853124 INFO (jetty-launcher-1416-thread-2) [n:127.0.0.1:64028_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 853126 INFO (jetty-launcher-1416-thread-2) [n:127.0.0.1:64028_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:64028_solr
[junit4] 2> 853126 INFO (zkCallback-1450-thread-1-processing-n:127.0.0.1:38378_solr) [n:127.0.0.1:38378_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 853141 INFO (zkCallback-1450-thread-1-processing-n:127.0.0.1:38378_solr) [n:127.0.0.1:38378_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 853141 INFO (zkCallback-1448-thread-1-processing-n:127.0.0.1:46272_solr) [n:127.0.0.1:46272_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 853141 INFO (zkCallback-1432-thread-1-processing-n:127.0.0.1:35438_solr) [n:127.0.0.1:35438_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 853148 INFO (zkCallback-1446-thread-1-processing-n:127.0.0.1:64028_solr) [n:127.0.0.1:64028_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 853160 INFO (jetty-launcher-1416-thread-4) [n:127.0.0.1:38378_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38378_solr
[junit4] 2> 853168 INFO (zkCallback-1448-thread-1-processing-n:127.0.0.1:46272_solr) [n:127.0.0.1:46272_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 853168 INFO (zkCallback-1450-thread-1-processing-n:127.0.0.1:38378_solr) [n:127.0.0.1:38378_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 853168 INFO (zkCallback-1432-thread-1-processing-n:127.0.0.1:35438_solr) [n:127.0.0.1:35438_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 853168 INFO (zkCallback-1446-thread-1-processing-n:127.0.0.1:64028_solr) [n:127.0.0.1:64028_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 853186 INFO (jetty-launcher-1416-thread-1) [n:127.0.0.1:46272_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46272.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2aca09da
[junit4] 2> 853194 INFO (jetty-launcher-1416-thread-1) [n:127.0.0.1:46272_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46272.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2aca09da
[junit4] 2> 853194 INFO (jetty-launcher-1416-thread-1) [n:127.0.0.1:46272_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46272.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2aca09da
[junit4] 2> 853196 INFO (jetty-launcher-1416-thread-1) [n:127.0.0.1:46272_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_48EB1A90529CBD0F-001/tempDir-001/node1/.
[junit4] 2> 853236 INFO (jetty-launcher-1416-thread-2) [n:127.0.0.1:64028_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64028.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2aca09da
[junit4] 2> 853248 INFO (jetty-launcher-1416-thread-2) [n:127.0.0.1:64028_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64028.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2aca09da
[junit4] 2> 853248 INFO (jetty-launcher-1416-thread-2) [n:127.0.0.1:64028_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64028.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2aca09da
[junit4] 2> 853249 INFO (jetty-launcher-1416-thread-2) [n:127.0.0.1:64028_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_48EB1A90529CBD0F-001/tempDir-001/node2/.
[junit4] 2> 853278 INFO (jetty-launcher-1416-thread-4) [n:127.0.0.1:38378_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38378.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2aca09da
[junit4] 2> 853285 INFO (jetty-launcher-1416-thread-4) [n:127.0.0.1:38378_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38378.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2aca09da
[junit4] 2> 853286 INFO (jetty-launcher-1416-thread-4) [n:127.0.0.1:38378_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38378.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2aca09da
[junit4] 2> 853287 INFO (jetty-launcher-1416-thread-4) [n:127.0.0.1:38378_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_48EB1A90529CBD0F-001/tempDir-001/node4/.
[junit4] 2> 853379 INFO (jetty-launcher-1416-thread-3) [n:127.0.0.1:35438_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35438.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2aca09da
[junit4] 2> 853389 INFO (jetty-launcher-1416-thread-3) [n:127.0.0.1:35438_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35438.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2aca09da
[junit4] 2> 853390 INFO (jetty-launcher-1416-thread-3) [n:127.0.0.1:35438_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35438.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2aca09da
[junit4] 2> 853390 INFO (jetty-launcher-1416-thread-3) [n:127.0.0.1:35438_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_48EB1A90529CBD0F-001/tempDir-001/node3/.
[junit4] 2> 853431 INFO (zkConnectionManagerCallback-1461-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 853435 INFO (zkConnectionManagerCallback-1465-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 853437 INFO (SUITE-MoveReplicaHDFSTest-seed#[48EB1A90529CBD0F]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 853439 INFO (SUITE-MoveReplicaHDFSTest-seed#[48EB1A90529CBD0F]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34090/solr ready
[junit4] 2> 853450 INFO (qtp1191384141-7556) [n:127.0.0.1:35438_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :overseerstatus with params action=OVERSEERSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 853463 INFO (qtp1191384141-7556) [n:127.0.0.1:35438_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=OVERSEERSTATUS&wt=javabin&version=2} status=0 QTime=13
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 853487 WARN (SUITE-MoveReplicaHDFSTest-seed#[48EB1A90529CBD0F]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 853492 WARN (SUITE-MoveReplicaHDFSTest-seed#[48EB1A90529CBD0F]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 853493 INFO (SUITE-MoveReplicaHDFSTest-seed#[48EB1A90529CBD0F]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 853506 INFO (SUITE-MoveReplicaHDFSTest-seed#[48EB1A90529CBD0F]-worker) [ ] o.m.log Extract jar:file:/export/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/hdfs to ./temp/Jetty_serv1.vm3_39116_hdfs____.7qrw7j/webapp
[junit4] 2> 854001 INFO (SUITE-MoveReplicaHDFSTest-seed#[48EB1A90529CBD0F]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@serv1-vm3:39116
[junit4] 2> 854151 WARN (SUITE-MoveReplicaHDFSTest-seed#[48EB1A90529CBD0F]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 854152 INFO (SUITE-MoveReplicaHDFSTest-seed#[48EB1A90529CBD0F]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 854162 INFO (SUITE-MoveReplicaHDFSTest-seed#[48EB1A90529CBD0F]-worker) [ ] o.m.log Extract jar:file:/export/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_40158_datanode____sh8jgf/webapp
[junit4] 2> 854755 INFO (SUITE-MoveReplicaHDFSTest-seed#[48EB1A90529CBD0F]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40158
[junit4] 2> 854932 WARN (SUITE-MoveReplicaHDFSTest-seed#[48EB1A90529CBD0F]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 854933 INFO (SUITE-MoveReplicaHDFSTest-seed#[48EB1A90529CBD0F]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 854961 INFO (SUITE-MoveReplicaHDFSTest-seed#[48EB1A90529CBD0F]-worker) [ ] o.m.log Extract jar:file:/export/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_38724_datanode____.9fb8it/webapp
[junit4] 2> 855040 ERROR (DataNode: [[[DISK]file:/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_48EB1A90529CBD0F-001/tempDir-002/hdfsBaseDir/data/data1/, [DISK]file:/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_48EB1A90529CBD0F-001/tempDir-002/hdfsBaseDir/data/data2/]] heartbeating to serv1-vm3/127.0.0.1:53134) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 855096 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x44deb24bb3242: from storage DS-d3864fe2-7d71-4e72-bff5-0ad44ae9deab node DatanodeRegistration(127.0.0.1:54576, datanodeUuid=da9f4fac-857f-4bcc-8107-44d688200088, infoPort=33618, infoSecurePort=0, ipcPort=45710, storageInfo=lv=-56;cid=testClusterID;nsid=202548131;c=0), blocks: 0, hasStaleStorage: true, processing time: 3 msecs
[junit4] 2> 855097 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x44deb24bb3242: from storage DS-4b489ee9-ecc4-46e8-bd16-e4503f1b4efa node DatanodeRegistration(127.0.0.1:54576, datanodeUuid=da9f4fac-857f-4bcc-8107-44d688200088, infoPort=33618, infoSecurePort=0, ipcPort=45710, storageInfo=lv=-56;cid=testClusterID;nsid=202548131;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 855461 INFO (OverseerCollectionConfigSetProcessor-72136995488464908-127.0.0.1:38378_solr-n_0000000000) [n:127.0.0.1:38378_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 855519 INFO (SUITE-MoveReplicaHDFSTest-seed#[48EB1A90529CBD0F]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:38724
[junit4] 2> 855628 ERROR (DataNode: [[[DISK]file:/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_48EB1A90529CBD0F-001/tempDir-002/hdfsBaseDir/data/data3/, [DISK]file:/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_48EB1A90529CBD0F-001/tempDir-002/hdfsBaseDir/data/data4/]] heartbeating to serv1-vm3/127.0.0.1:53134) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 855637 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x44deb45232b7c: from storage DS-329b9aa7-9b3e-4cab-a60a-5ccb691f91d7 node DatanodeRegistration(127.0.0.1:53513, datanodeUuid=00503c81-2091-4541-b2de-de71e56ebe7a, infoPort=51864, infoSecurePort=0, ipcPort=63771, storageInfo=lv=-56;cid=testClusterID;nsid=202548131;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 855637 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x44deb45232b7c: from storage DS-026a34a6-2dc6-4c85-8ed2-b9b23ae7f26b node DatanodeRegistration(127.0.0.1:53513, datanodeUuid=00503c81-2091-4541-b2de-de71e56ebe7a, infoPort=51864, infoSecurePort=0, ipcPort=63771, storageInfo=lv=-56;cid=testClusterID;nsid=202548131;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 855769 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[48EB1A90529CBD0F]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testFailedMove
[junit4] 2> 855771 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[48EB1A90529CBD0F]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 855773 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[48EB1A90529CBD0F]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T22:27:37+01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 855774 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[48EB1A90529CBD0F]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 855774 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[48EB1A90529CBD0F]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 855774 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[48EB1A90529CBD0F]) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 855774 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[48EB1A90529CBD0F]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@465afce3{/solr,null,AVAILABLE}
[junit4] 2> 855775 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[48EB1A90529CBD0F]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@4c6b9e59{HTTP/1.1,[http/1.1]}{127.0.0.1:57493}
[junit4] 2> 855775 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[48EB1A90529CBD0F]) [ ] o.e.j.s.Server Started @859294ms
[junit4] 2> 855775 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[48EB1A90529CBD0F]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=57493}
[junit4] 2> 855775 ERROR (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[48EB1A90529CBD0F]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 855775 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[48EB1A90529CBD0F]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 855775 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[48EB1A90529CBD0F]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 855775 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[48EB1A90529CBD0F]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 855775 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[48EB1A90529CBD0F]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-03T22:15:53.463Z
[junit4] 2> 855780 INFO (zkConnectionManagerCallback-1468-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 855781 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[48EB1A90529CBD0F]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 855790 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[48EB1A90529CBD0F]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34090/solr
[junit4] 2> 855794 INFO (zkConnectionManagerCallback-1472-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 855798 INFO (zkConnectionManagerCallback-1474-thread-1-processing-n:127.0.0.1:57493_solr) [n:127.0.0.1:57493_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 855812 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[48EB1A90529CBD0F]) [n:127.0.0.1:57493_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 855816 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[48EB1A90529CBD0F]) [n:127.0.0.1:57493_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 855820 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[48EB1A90529CBD0F]) [n:127.0.0.1:57493_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57493_solr
[junit4] 2> 855822 INFO (zkCallback-1448-thread-1-processing-n:127.0.0.1:46272_solr) [n:127.0.0.1:46272_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 855822 INFO (zkCallback-1450-thread-1-processing-n:127.0.0.1:38378_solr) [n:127.0.0.1:38378_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 855823 INFO (zkCallback-1432-thread-1-processing-n:127.0.0.1:35438_solr) [n:127.0.0.1:35438_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 855824 INFO (zkCallback-1464-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 855824 INFO (zkCallback-1464-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 855824 INFO (zkCallback-1446-thread-1-processing-n:127.0.0.1:64028_solr) [n:127.0.0.1:64028_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 855826 INFO (zkCallback-1473-thread-1-processing-n:127.0.0.1:57493_solr) [n:127.0.0.1:57493_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 855960 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[48EB1A90529CBD0F]) [n:127.0.0.1:57493_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_57493.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2aca09da
[junit4] 2> 855978 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[48EB1A90529CBD0F]) [n:127.0.0.1:57493_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_57493.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2aca09da
[junit4] 2> 855978 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[48EB1A90529CBD0F]) [n:127.0.0.1:57493_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_57493.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2aca09da
[junit4] 2> 855979 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[48EB1A90529CBD0F]) [n:127.0.0.1:57493_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_48EB1A90529CBD0F-001/tempDir-001/node5/.
[junit4] 2> 856027 INFO (zkConnectionManagerCallback-1478-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 856031 INFO (qtp1191384141-7559) [n:127.0.0.1:35438_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf1&autoAddReplicas=false&name=MoveReplicaHDFSTest_failed_coll_true&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 856033 INFO (OverseerThreadFactory-2873-thread-2-processing-n:127.0.0.1:38378_solr) [n:127.0.0.1:38378_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection MoveReplicaHDFSTest_failed_coll_true
[junit4] 2> 856146 INFO (OverseerStateUpdate-72136995488464908-127.0.0.1:38378_solr-n_0000000000) [n:127.0.0.1:38378_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"MoveReplicaHDFSTest_failed_coll_true",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:46272/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 856150 INFO (OverseerStateUpdate-72136995488464908-127.0.0.1:38378_solr-n_0000000000) [n:127.0.0.1:38378_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"MoveReplicaHDFSTest_failed_coll_true",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:64028/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 856154 INFO (OverseerStateUpdate-72136995488464908-127.0.0.1:38378_solr-n_0000000000) [n:127.0.0.1:38378_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"MoveReplicaHDFSTest_failed_coll_true",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:38378/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 856158 INFO (OverseerStateUpdate-72136995488464908-127.0.0.1:38378_solr-n_0000000000) [n:127.0.0.1:38378_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"MoveReplicaHDFSTest_failed_coll_true",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:57493/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 856365 INFO (qtp1280834553-7792) [n:127.0.0.1:57493_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf1&newCollection=true&name=MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6&action=CREATE&numShards=2&collection=MoveReplicaHDFSTest_failed_coll_true&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 856365 INFO (qtp1280834553-7792) [n:127.0.0.1:57493_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 856366 INFO (qtp1161599098-7536) [n:127.0.0.1:38378_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf1&newCollection=true&name=MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n4&action=CREATE&numShards=2&collection=MoveReplicaHDFSTest_failed_coll_true&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 856367 INFO (qtp1161599098-7536) [n:127.0.0.1:38378_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 856367 INFO (qtp777293901-7544) [n:127.0.0.1:46272_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf1&newCollection=true&name=MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1&action=CREATE&numShards=2&collection=MoveReplicaHDFSTest_failed_coll_true&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 856368 INFO (qtp777293901-7544) [n:127.0.0.1:46272_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 856369 INFO (qtp193480918-7528) [n:127.0.0.1:64028_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=conf1&newCollection=true&name=MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2&action=CREATE&numShards=2&collection=MoveReplicaHDFSTest_failed_coll_true&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 856369 INFO (qtp193480918-7528) [n:127.0.0.1:64028_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 856491 INFO (zkCallback-1448-thread-1-processing-n:127.0.0.1:46272_solr) [n:127.0.0.1:46272_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_failed_coll_true/state.json] for collection [MoveReplicaHDFSTest_failed_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 856491 INFO (zkCallback-1450-thread-1-processing-n:127.0.0.1:38378_solr) [n:127.0.0.1:38378_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_failed_coll_true/state.json] for collection [MoveReplicaHDFSTest_failed_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 856491 INFO (zkCallback-1448-thread-2-processing-n:127.0.0.1:46272_solr) [n:127.0.0.1:46272_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_failed_coll_true/state.json] for collection [MoveReplicaHDFSTest_failed_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 856491 INFO (zkCallback-1446-thread-1-processing-n:127.0.0.1:64028_solr) [n:127.0.0.1:64028_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_failed_coll_true/state.json] for collection [MoveReplicaHDFSTest_failed_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 856492 INFO (zkCallback-1473-thread-2-processing-n:127.0.0.1:57493_solr) [n:127.0.0.1:57493_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_failed_coll_true/state.json] for collection [MoveReplicaHDFSTest_failed_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 856492 INFO (zkCallback-1473-thread-1-processing-n:127.0.0.1:57493_solr) [n:127.0.0.1:57493_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_failed_coll_true/state.json] for collection [MoveReplicaHDFSTest_failed_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 856492 INFO (zkCallback-1450-thread-2-processing-n:127.0.0.1:38378_solr) [n:127.0.0.1:38378_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_failed_coll_true/state.json] for collection [MoveReplicaHDFSTest_failed_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 856492 INFO (zkCallback-1446-thread-2-processing-n:127.0.0.1:64028_solr) [n:127.0.0.1:64028_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_failed_coll_true/state.json] for collection [MoveReplicaHDFSTest_failed_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 857394 INFO (qtp1161599098-7536) [n:127.0.0.1:38378_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 857404 INFO (qtp777293901-7544) [n:127.0.0.1:46272_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 857414 INFO (qtp1161599098-7536) [n:127.0.0.1:38378_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n4] o.a.s.s.IndexSchema [MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n4] Schema name=minimal
[junit4] 2> 857416 INFO (qtp1161599098-7536) [n:127.0.0.1:38378_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n4] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 857416 INFO (qtp1161599098-7536) [n:127.0.0.1:38378_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n4] o.a.s.c.CoreContainer Creating SolrCore 'MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n4' using configuration from collection MoveReplicaHDFSTest_failed_coll_true, trusted=true
[junit4] 2> 857416 INFO (qtp1161599098-7536) [n:127.0.0.1:38378_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n4] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38378.solr.core.MoveReplicaHDFSTest_failed_coll_true.shard2.replica_n4' (registry 'solr.core.MoveReplicaHDFSTest_failed_coll_true.shard2.replica_n4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2aca09da
[junit4] 2> 857417 INFO (qtp1161599098-7536) [n:127.0.0.1:38378_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n4] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://serv1-vm3:53134/data
[junit4] 2> 857417 INFO (qtp1161599098-7536) [n:127.0.0.1:38378_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n4] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 857417 INFO (qtp1161599098-7536) [n:127.0.0.1:38378_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n4] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 857417 INFO (qtp1161599098-7536) [n:127.0.0.1:38378_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n4] o.a.s.c.SolrCore [[MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n4] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_48EB1A90529CBD0F-001/tempDir-001/node4/MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n4], dataDir=[hdfs://serv1-vm3:53134/data/MoveReplicaHDFSTest_failed_coll_true/core_node7/data/]
[junit4] 2> 857417 INFO (qtp193480918-7528) [n:127.0.0.1:64028_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 857418 INFO (qtp1161599098-7536) [n:127.0.0.1:38378_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n4] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://serv1-vm3:53134/data/MoveReplicaHDFSTest_failed_coll_true/core_node7/data/snapshot_metadata
[junit4] 2> 857432 INFO (qtp193480918-7528) [n:127.0.0.1:64028_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.s.IndexSchema [MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] Schema name=minimal
[junit4] 2> 857440 INFO (qtp193480918-7528) [n:127.0.0.1:64028_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 857440 INFO (qtp193480918-7528) [n:127.0.0.1:64028_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2' using configuration from collection MoveReplicaHDFSTest_failed_coll_true, trusted=true
[junit4] 2> 857440 INFO (qtp193480918-7528) [n:127.0.0.1:64028_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64028.solr.core.MoveReplicaHDFSTest_failed_coll_true.shard1.replica_n2' (registry 'solr.core.MoveReplicaHDFSTest_failed_coll_true.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2aca09da
[junit4] 2> 857441 INFO (qtp193480918-7528) [n:127.0.0.1:64028_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://serv1-vm3:53134/data
[junit4] 2> 857441 INFO (qtp193480918-7528) [n:127.0.0.1:64028_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 857441 INFO (qtp193480918-7528) [n:127.0.0.1:64028_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 857441 INFO (qtp193480918-7528) [n:127.0.0.1:64028_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.c.SolrCore [[MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_48EB1A90529CBD0F-001/tempDir-001/node2/MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2], dataDir=[hdfs://serv1-vm3:53134/data/MoveReplicaHDFSTest_failed_coll_true/core_node5/data/]
[junit4] 2> 857442 INFO (qtp193480918-7528) [n:127.0.0.1:64028_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://serv1-vm3:53134/data/MoveReplicaHDFSTest_failed_coll_true/core_node5/data/snapshot_metadata
[junit4] 2> 857442 INFO (qtp777293901-7544) [n:127.0.0.1:46272_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] o.a.s.s.IndexSchema [MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] Schema name=minimal
[junit4] 2> 857449 INFO (qtp777293901-7544) [n:127.0.0.1:46272_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 857449 INFO (qtp777293901-7544) [n:127.0.0.1:46272_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1' using configuration from collection MoveReplicaHDFSTest_failed_coll_true, trusted=true
[junit4] 2> 857450 INFO (qtp777293901-7544) [n:127.0.0.1:46272_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46272.solr.core.MoveReplicaHDFSTest_failed_coll_true.shard1.replica_n1' (registry 'solr.core.MoveReplicaHDFSTest_failed_coll_true.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2aca09da
[junit4] 2> 857450 INFO (qtp777293901-7544) [n:127.0.0.1:46272_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://serv1-vm3:53134/data
[junit4] 2> 857450 INFO (qtp777293901-7544) [n:127.0.0.1:46272_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 857450 INFO (qtp777293901-7544) [n:127.0.0.1:46272_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 857450 INFO (qtp777293901-7544) [n:127.0.0.1:46272_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] o.a.s.c.SolrCore [[MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_48EB1A90529CBD0F-001/tempDir-001/node1/MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1], dataDir=[hdfs://serv1-vm3:53134/data/MoveReplicaHDFSTest_failed_coll_true/core_node3/data/]
[junit4] 2> 857451 INFO (qtp777293901-7544) [n:127.0.0.1:46272_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://serv1-vm3:53134/data/MoveReplicaHDFSTest_failed_coll_true/core_node3/data/snapshot_metadata
[junit4] 2> 857464 INFO (qtp193480918-7528) [n:127.0.0.1:64028_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://serv1-vm3:53134/data/MoveReplicaHDFSTest_failed_coll_true/core_node5/data
[junit4] 2> 857467 INFO (qtp1280834553-7792) [n:127.0.0.1:57493_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 857477 INFO (qtp1280834553-7792) [n:127.0.0.1:57493_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.s.IndexSchema [MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] Schema name=minimal
[junit4] 2> 857479 INFO (qtp1280834553-7792) [n:127.0.0.1:57493_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 857480 INFO (qtp1280834553-7792) [n:127.0.0.1:57493_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.c.CoreContainer Creating SolrCore 'MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6' using configuration from collection MoveReplicaHDFSTest_failed_coll_true, trusted=true
[junit4] 2> 857480 INFO (qtp1280834553-7792) [n:127.0.0.1:57493_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_57493.solr.core.MoveReplicaHDFSTest_failed_coll_true.shard2.replica_n6' (registry 'solr.core.MoveReplicaHDFSTest_failed_coll_true.shard2.replica_n6') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2aca09da
[junit4] 2> 857480 INFO (qtp1280834553-7792) [n:127.0.0.1:57493_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://serv1-vm3:53134/data
[junit4] 2> 857480 INFO (qtp1280834553-7792) [n:127.0.0.1:57493_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 857480 INFO (qtp1280834553-7792) [n:127.0.0.1:57493_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 857480 INFO (qtp1280834553-7792) [n:127.0.0.1:57493_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.c.SolrCore [[MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.MoveReplicaHDFSTest_48EB1A90529CBD0F-001/tempDir-001/node5/MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6], dataDir=[hdfs://serv1-vm3:53134/data/MoveReplicaHDFSTest_failed_coll_true/core_node8/data/]
[junit4] 2> 857485 INFO (qtp1161599098-7536) [n:127.0.0.1:38378_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n4] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://serv1-vm3:53134/data/MoveReplicaHDFSTest_failed_coll_true/core_node7/data
[junit4] 2> 857485 INFO (qtp1280834553-7792) [n:127.0.0.1:57493_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://serv1-vm3:53134/data/MoveReplicaHDFSTest_failed_coll_true/core_node8/data/snapshot_metadata
[junit4] 2> 857495 INFO (qtp777293901-7544) [n:127.0.0.1:46272_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://serv1-vm3:53134/data/MoveReplicaHDFSTest_failed_coll_true/core_node3/data
[junit4] 2> 857502 INFO (qtp193480918-7528) [n:127.0.0.1:64028_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://serv1-vm3:53134/data/MoveReplicaHDFSTest_failed_coll_true/core_node5/data/index
[junit4] 2> 857503 INFO (qtp1280834553-7792) [n:127.0.0.1:57493_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://serv1-vm3:53134/data/MoveReplicaHDFSTest_failed_coll_true/core_node8/data
[junit4] 2> 857511 INFO (qtp777293901-7544) [n:127.0.0.1:46272_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://serv1-vm3:53134/data/MoveReplicaHDFSTest_failed_coll_true/core_node3/data/index
[junit4] 2> 857525 INFO (qtp1161599098-7536) [n:127.0.0.1:38378_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n4] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://serv1-vm3:53134/data/MoveReplicaHDFSTest_failed_coll_true/core_node7/data/index
[junit4] 2> 857528 INFO (qtp1280834553-7792) [n:127.0.0.1:57493_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://serv1-vm3:53134/data/MoveReplicaHDFSTest_failed_coll_true/core_node8/data/index
[junit4] 2> 857584 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:54576 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-329b9aa7-9b3e-4cab-a60a-5ccb691f91d7:NORMAL:127.0.0.1:53513|RBW], ReplicaUC[[DISK]DS-4b489ee9-ecc4-46e8-bd16-e4503f1b4efa:NORMAL:127.0.0.1:54576|RBW]]} size 0
[junit4] 2> 857585 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:53513 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-329b9aa7-9b3e-4cab-a60a-5ccb691f91d7:NORMAL:127.0.0.1:53513|RBW], ReplicaUC[[DISK]DS-4b489ee9-ecc4-46e8-bd16-e4503f1b4efa:NORMAL:127.0.0.1:54576|RBW]]} size 0
[junit4] 2> 857598 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:54576 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-026a34a6-2dc6-4c85-8ed2-b9b23ae7f26b:NORMAL:127.0.0.1:53513|RBW], ReplicaUC[[DISK]DS-d3864fe2-7d71-4e72-bff5-0ad44ae9deab:NORMAL:127.0.0.1:54576|FINALIZED]]} size 0
[junit4] 2> 857599 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:53513 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-026a34a6-2dc6-4c85-8ed2-b9b23ae7f26b:NORMAL:127.0.0.1:53513|RBW], ReplicaUC[[DISK]DS-d3864fe2-7d71-4e72-bff5-0ad44ae9deab:NORMAL:127.0.0.1:54576|FINALIZED]]} size 0
[junit4] 2> 857759 INFO (qtp193480918-7528) [n:127.0.0.1:64028_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 857759 INFO (qtp193480918-7528) [n:127.0.0.1:64028_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 857759 INFO (qtp193480918-7528) [n:127.0.0.1:64028_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
[junit4] 2> 857767 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:53513 is added to blk_1073741827_1003{UCState=COMMITTED, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-4b489ee9-ecc4-46e8-bd16-e4503f1b4efa:NORMAL:127.0.0.1:54576|RBW], ReplicaUC[[DISK]DS-026a34a6-2dc6-4c85-8ed2-b9b23ae7f26b:NORMAL:127.0.0.1:53513|RBW]]} size 69
[junit4] 2> 857767 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:54576 is added to blk_1073741827_1003 size 69
[junit4] 2> 857781 INFO (qtp1280834553-7792) [n:127.0.0.1:57493_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 857781 INFO (qtp1280834553-7792) [n:127.0.0.1:57493_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 857781 INFO (qtp1280834553-7792) [n:127.0.0.1:57493_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
[junit4] 2> 857788 INFO (qtp193480918-7528) [n:127.0.0.1:64028_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 857788 INFO (qtp193480918-7528) [n:127.0.0.1:64028_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 857791 INFO (qtp1280834553-7792) [n:127.0.0.1:57493_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 857791 INFO (qtp1280834553-7792) [n:127.0.0.1:57493_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 857815 INFO (qtp1280834553-7792) [n:127.0.0.1:57493_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.s.SolrIndexSearcher Opening [Searcher@5c784fca[MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] main]
[junit4] 2> 857817 INFO (qtp1280834553-7792) [n:127.0.0.1:57493_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 857818 INFO (qtp1280834553-7792) [n:127.0.0.1:57493_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 857818 INFO (qtp1280834553-7792) [n:127.0.0.1:57493_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 857819 INFO (qtp1280834553-7792) [n:127.0.0.1:57493_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1591419763956908032
[junit4] 2> 857844 INFO (searcherExecutor-2890-thread-1-processing-n:127.0.0.1:57493_solr x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6 s:shard2 c:MoveReplicaHDFSTest_failed_coll_true r:core_node8) [n:127.0.0.1:57493_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.c.SolrCore [MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] Registered new searcher Searcher@5c784fca[MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 857850 INFO (qtp1280834553-7792) [n:127.0.0.1:57493_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard2: total=2 found=1 timeoutin=9999ms
[junit4] 2> 857854 INFO (qtp193480918-7528) [n:127.0.0.1:64028_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@1715c659[MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] main]
[junit4] 2> 857856 INFO (qtp193480918-7528) [n:127.0.0.1:64028_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.r.ManagedResourceStorage C
[...truncated too long message...]
r.jetty, tag=null
[junit4] 2> 229018 INFO (TEST-SchemaTest.copyFieldsShouldFailWhenOneOfTheFieldsDoesntExistInTheSchema-seed#[2B21A3CEC8AAF2DD]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1ef163f2: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@53d4436a
[junit4] 2> 229020 INFO (coreCloseExecutor-1260-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@39e1d7a5
[junit4] 2> 229020 INFO (coreCloseExecutor-1260-thread-1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=971102117
[junit4] 2> 229020 INFO (coreCloseExecutor-1260-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@163c77e1: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@6ba20d1
[junit4] 2> 229028 INFO (TEST-SchemaTest.copyFieldsShouldFailWhenOneOfTheFieldsDoesntExistInTheSchema-seed#[2B21A3CEC8AAF2DD]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@e46cbc5{/solr,null,UNAVAILABLE}
[junit4] 2> 229028 INFO (TEST-SchemaTest.copyFieldsShouldFailWhenOneOfTheFieldsDoesntExistInTheSchema-seed#[2B21A3CEC8AAF2DD]) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 229058 INFO (TEST-SchemaTest.copyFieldsShouldFailWhenOneOfTheFieldsDoesntExistInTheSchema-seed#[2B21A3CEC8AAF2DD]) [ ] o.a.s.SolrTestCaseJ4 ###Ending copyFieldsShouldFailWhenOneOfTheFieldsDoesntExistInTheSchema
[junit4] 2> 229059 INFO (SUITE-SchemaTest-seed#[2B21A3CEC8AAF2DD]-worker) [ ] o.a.s.SolrTestCaseJ4 ------------------------------------------------------- Done waiting for tracked resources to be released
[junit4] 2> Feb 03, 2018 11:21:58 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> Feb 03, 2018 11:22:18 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.solr.client.solrj.request.SchemaTest:
[junit4] 2> 1) Thread[id=3235, name=qtp1073869087-3235, state=TIMED_WAITING, group=TGRP-SchemaTest]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Feb 03, 2018 11:22:18 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=3235, name=qtp1073869087-3235, state=TIMED_WAITING, group=TGRP-SchemaTest]
[junit4] 2> Feb 03, 2018 11:22:21 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> SEVERE: There are still zombie threads that couldn't be terminated:
[junit4] 2> 1) Thread[id=3235, name=qtp1073869087-3235, state=TIMED_WAITING, group=TGRP-SchemaTest]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=1009, maxMBSortInHeap=7.213368036505777, sim=RandomSimilarity(queryNorm=false): {}, locale=sq-AL, timezone=Etc/GMT-3
[junit4] 2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=3,threads=2,free=29987784,total=159641600
[junit4] 2> NOTE: All tests run in this JVM: [GreaterThanEvaluatorTest, JettyWebappTest, TestDelegationTokenResponse, ArrayEvaluatorTest, TestEmbeddedSolrServer, TestPolicy, FieldAnalysisResponseTest, CosineEvaluatorTest, NamedListTest, SubtractEvaluatorTest, TestRetryUtil, StreamingTest, MultiplyEvaluatorTest, LengthEvaluatorTest, ModifiableSolrParamsTest, SolrExampleEmbeddedTest, TestV1toV2ApiMapper, TestCloudSolrClientConnections, ConcatOperationTest, ArcSineEvaluatorTest, EmpiricalDistributionEvaluatorTest, TermsResponseTest, AppendEvaluatorTest, NoOpResponseParserTest, NormalDistributionEvaluatorTest, TestUpdateRequestCodec, TestHash, HttpSolrClientSSLAuthConPoolTest, ConcurrentUpdateSolrClientBuilderTest, ShardParamsTest, TestSpellCheckResponse, TestToleratedUpdateError, AddEvaluatorTest, LessThanEvaluatorTest, CumulativeProbabilityEvaluatorTest, RecursiveEvaluatorTest, SolrExampleStreamingTest, SolrExampleBinaryTest, SolrExampleXMLTest, LargeVolumeBinaryJettyTest, TestBatchUpdate, LargeVolumeEmbeddedTest, LargeVolumeJettyTest, TestSolrProperties, SolrParamTest, QueryResponseTest, TestCoreAdmin, TestJavaBinCodec, TestDocumentObjectBinder, SolrQueryTest, TestXMLEscaping, DocumentAnalysisResponseTest, SolrDocumentTest, SolrExceptionTest, ClientUtilsTest, UsingSolrJRefGuideExamplesTest, CollectionAdminRequestRequiredParamsTest, GetByIdTest, SolrSchemalessExampleTest, TestLBHttpSolrClient, HttpSolrClientBuilderTest, HttpSolrClientConPoolTest, LBHttpSolrClientBuilderTest, GraphExpressionTest, JdbcTest, StreamExpressionToExpessionTest, AbsoluteValueEvaluatorTest, AscEvaluatorTest, CeilingEvaluatorTest, CoalesceEvaluatorTest, CubedRootEvaluatorTest, DivideEvaluatorTest, EqualToEvaluatorTest, ExclusiveOrEvaluatorTest, FloorEvaluatorTest, HyperbolicCosineEvaluatorTest, HyperbolicSineEvaluatorTest, HyperbolicTangentEvaluatorTest, NaturalLogEvaluatorTest, NotEvaluatorTest, OrEvaluatorTest, PowerEvaluatorTest, RawValueEvaluatorTest, ReverseEvaluatorTest, RoundEvaluatorTest, SineEvaluatorTest, UuidEvaluatorTest, SchemaTest]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=SchemaTest -Dtests.seed=2B21A3CEC8AAF2DD -Dtests.slow=true -Dtests.locale=sq-AL -Dtests.timezone=Etc/GMT-3 -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] ERROR 0.00s J0 | SchemaTest (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.client.solrj.request.SchemaTest:
[junit4] > 1) Thread[id=3235, name=qtp1073869087-3235, state=TIMED_WAITING, group=TGRP-SchemaTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([2B21A3CEC8AAF2DD]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=3235, name=qtp1073869087-3235, state=TIMED_WAITING, group=TGRP-SchemaTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([2B21A3CEC8AAF2DD]:0)
[junit4] Completed [133/146 (1!)] on J0 in 70.45s, 35 tests, 2 errors <<< FAILURES!
[...truncated 47327 lines...]