You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2018/12/11 11:05:57 UTC
[JENKINS] Lucene-Solr-7.x-Solaris (64bit/jdk1.8.0) - Build # 950 -
Failure!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Solaris/950/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseParallelGC
5 tests failed.
FAILED: org.apache.solr.cloud.MultiThreadedOCPTest.test
Error Message:
Captured an uncaught exception in thread: Thread[id=33904, name=OverseerThreadFactory-10809-thread-5, state=RUNNABLE, group=Overseer collection creation process.]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=33904, name=OverseerThreadFactory-10809-thread-5, state=RUNNABLE, group=Overseer collection creation process.]
at __randomizedtesting.SeedInfo.seed([3C5AC9B5D62D852E:B40EF66F78D1E8D6]:0)
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
at __randomizedtesting.SeedInfo.seed([3C5AC9B5D62D852E]:0)
at java.util.Collections$UnmodifiableMap.entrySet(Collections.java:1481)
at java.util.HashMap.putMapEntries(HashMap.java:512)
at java.util.HashMap.<init>(HashMap.java:490)
at org.apache.logging.log4j.spi.DefaultThreadContextMap.getCopy(DefaultThreadContextMap.java:186)
at org.apache.logging.log4j.ThreadContext.getContext(ThreadContext.java:343)
at org.apache.logging.slf4j.Log4jMDCAdapter.getCopyOfContextMap(Log4jMDCAdapter.java:51)
at org.slf4j.MDC.getCopyOfContextMap(MDC.java:248)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:169)
at com.codahale.metrics.InstrumentedExecutorService.execute(InstrumentedExecutorService.java:61)
at java.util.concurrent.ExecutorCompletionService.submit(ExecutorCompletionService.java:181)
at org.apache.solr.handler.component.HttpShardHandler.submit(HttpShardHandler.java:203)
at org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.waitForCoreAdminAsyncCallToComplete(OverseerCollectionMessageHandler.java:926)
at org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.waitForAsyncCallsToComplete(OverseerCollectionMessageHandler.java:906)
at org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.processResponses(OverseerCollectionMessageHandler.java:776)
at org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.processResponses(OverseerCollectionMessageHandler.java:752)
at org.apache.solr.cloud.api.collections.AddReplicaCmd.lambda$addReplica$1(AddReplicaCmd.java:168)
at org.apache.solr.cloud.api.collections.AddReplicaCmd$$Lambda$666/1199632155.run(Unknown Source)
at org.apache.solr.cloud.api.collections.AddReplicaCmd.addReplica(AddReplicaCmd.java:190)
at org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.addReplica(OverseerCollectionMessageHandler.java:747)
at org.apache.solr.cloud.api.collections.SplitShardCmd.split(SplitShardCmd.java:280)
at org.apache.solr.cloud.api.collections.SplitShardCmd.call(SplitShardCmd.java:94)
at org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:292)
at org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:496)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$9/557611722.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.metrics.reporters.SolrJmxReporterCloudTest
Error Message:
Clean up static fields (in @AfterClass?), your test seems to hang on to approximately 10,753,088 bytes (threshold is 10,485,760). Field reference sizes (counted individually): - 10,752,104 bytes, private static javax.management.MBeanServer org.apache.solr.metrics.reporters.SolrJmxReporterCloudTest.mBeanServer - 296 bytes, public static org.junit.rules.TestRule org.apache.solr.SolrTestCaseJ4.solrClassRules - 152 bytes, private static java.util.Map org.apache.solr.SolrTestCaseJ4.savedClassLogLevels - 144 bytes, private static java.lang.String org.apache.solr.SolrTestCaseJ4.factoryProp - 128 bytes, private static java.lang.String org.apache.solr.metrics.reporters.SolrJmxReporterCloudTest.COLLECTION - 120 bytes, protected static volatile java.util.concurrent.ExecutorService org.apache.solr.SolrTestCaseJ4.testExecutor - 80 bytes, protected static java.lang.String org.apache.solr.SolrTestCaseJ4.coreName - 64 bytes, private static java.lang.String org.apache.solr.SolrTestCaseJ4.initialRootLogLevel
Stack Trace:
junit.framework.AssertionFailedError: Clean up static fields (in @AfterClass?), your test seems to hang on to approximately 10,753,088 bytes (threshold is 10,485,760). Field reference sizes (counted individually):
- 10,752,104 bytes, private static javax.management.MBeanServer org.apache.solr.metrics.reporters.SolrJmxReporterCloudTest.mBeanServer
- 296 bytes, public static org.junit.rules.TestRule org.apache.solr.SolrTestCaseJ4.solrClassRules
- 152 bytes, private static java.util.Map org.apache.solr.SolrTestCaseJ4.savedClassLogLevels
- 144 bytes, private static java.lang.String org.apache.solr.SolrTestCaseJ4.factoryProp
- 128 bytes, private static java.lang.String org.apache.solr.metrics.reporters.SolrJmxReporterCloudTest.COLLECTION
- 120 bytes, protected static volatile java.util.concurrent.ExecutorService org.apache.solr.SolrTestCaseJ4.testExecutor
- 80 bytes, protected static java.lang.String org.apache.solr.SolrTestCaseJ4.coreName
- 64 bytes, private static java.lang.String org.apache.solr.SolrTestCaseJ4.initialRootLogLevel
at __randomizedtesting.SeedInfo.seed([3C5AC9B5D62D852E]:0)
at com.carrotsearch.randomizedtesting.rules.StaticFieldsInvariantRule$1.afterAlways(StaticFieldsInvariantRule.java:170)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.common.cloud.SolrZkClientTest.testSimpleUpdateACLs
Error Message:
java.lang.InterruptedException: sleep interrupted
Stack Trace:
java.lang.RuntimeException: java.lang.InterruptedException: sleep interrupted
at org.apache.solr.cloud.ZkTestServer.run(ZkTestServer.java:600)
at org.apache.solr.cloud.ZkTestServer.run(ZkTestServer.java:524)
at org.apache.solr.common.cloud.SolrZkClientTest.setUp(SolrZkClientTest.java:61)
at sun.reflect.GeneratedMethodAccessor1.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:969)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.InterruptedException: sleep interrupted
at java.lang.Thread.sleep(Native Method)
at org.apache.solr.cloud.ZkTestServer.run(ZkTestServer.java:582)
... 39 more
FAILED: junit.framework.TestSuite.org.apache.solr.common.cloud.SolrZkClientTest
Error Message:
5 threads leaked from SUITE scope at org.apache.solr.common.cloud.SolrZkClientTest: 1) Thread[id=6626, name=SyncThread:0, state=WAITING, group=TGRP-SolrZkClientTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:127) 2) Thread[id=6627, name=ProcessThread(sid:0 cport:49726):, state=WAITING, group=TGRP-SolrZkClientTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:123) 3) Thread[id=6625, name=SessionTracker, state=TIMED_WAITING, group=TGRP-SolrZkClientTest] at java.lang.Object.wait(Native Method) at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:147) 4) Thread[id=6624, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-SolrZkClientTest] at sun.nio.ch.DevPollArrayWrapper.poll0(Native Method) at sun.nio.ch.DevPollArrayWrapper.poll(DevPollArrayWrapper.java:223) at sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:98) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:196) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=6623, name=ZkTestServer Run Thread, state=WAITING, group=TGRP-SolrZkClientTest] at java.lang.Object.wait(Native Method) at java.lang.Thread.join(Thread.java:1252) at java.lang.Thread.join(Thread.java:1326) at org.apache.zookeeper.server.NIOServerCnxnFactory.join(NIOServerCnxnFactory.java:313) at org.apache.solr.cloud.ZkTestServer$ZKServerMain.runFromConfig(ZkTestServer.java:343) at org.apache.solr.cloud.ZkTestServer$2.run(ZkTestServer.java:564)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 5 threads leaked from SUITE scope at org.apache.solr.common.cloud.SolrZkClientTest:
1) Thread[id=6626, name=SyncThread:0, state=WAITING, group=TGRP-SolrZkClientTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:127)
2) Thread[id=6627, name=ProcessThread(sid:0 cport:49726):, state=WAITING, group=TGRP-SolrZkClientTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:123)
3) Thread[id=6625, name=SessionTracker, state=TIMED_WAITING, group=TGRP-SolrZkClientTest]
at java.lang.Object.wait(Native Method)
at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:147)
4) Thread[id=6624, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-SolrZkClientTest]
at sun.nio.ch.DevPollArrayWrapper.poll0(Native Method)
at sun.nio.ch.DevPollArrayWrapper.poll(DevPollArrayWrapper.java:223)
at sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:98)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:196)
at java.lang.Thread.run(Thread.java:748)
5) Thread[id=6623, name=ZkTestServer Run Thread, state=WAITING, group=TGRP-SolrZkClientTest]
at java.lang.Object.wait(Native Method)
at java.lang.Thread.join(Thread.java:1252)
at java.lang.Thread.join(Thread.java:1326)
at org.apache.zookeeper.server.NIOServerCnxnFactory.join(NIOServerCnxnFactory.java:313)
at org.apache.solr.cloud.ZkTestServer$ZKServerMain.runFromConfig(ZkTestServer.java:343)
at org.apache.solr.cloud.ZkTestServer$2.run(ZkTestServer.java:564)
at __randomizedtesting.SeedInfo.seed([C23229FC9129A0C6]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.common.cloud.SolrZkClientTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=6624, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-SolrZkClientTest] at sun.nio.ch.DevPollArrayWrapper.interrupt(Native Method) at sun.nio.ch.DevPollArrayWrapper.interrupt(DevPollArrayWrapper.java:300) at sun.nio.ch.DevPollSelectorImpl.wakeup(DevPollSelectorImpl.java:207) at java.nio.channels.spi.AbstractSelector$1.interrupt(AbstractSelector.java:213) at java.nio.channels.spi.AbstractSelector.begin(AbstractSelector.java:219) at sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:97) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:196) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=6624, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-SolrZkClientTest]
at sun.nio.ch.DevPollArrayWrapper.interrupt(Native Method)
at sun.nio.ch.DevPollArrayWrapper.interrupt(DevPollArrayWrapper.java:300)
at sun.nio.ch.DevPollSelectorImpl.wakeup(DevPollSelectorImpl.java:207)
at java.nio.channels.spi.AbstractSelector$1.interrupt(AbstractSelector.java:213)
at java.nio.channels.spi.AbstractSelector.begin(AbstractSelector.java:219)
at sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:97)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:196)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([C23229FC9129A0C6]:0)
Build Log:
[...truncated 14160 lines...]
[junit4] Suite: org.apache.solr.cloud.MultiThreadedOCPTest
[junit4] 2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/init-core-data-001
[junit4] 2> 1891595 WARN (SUITE-MultiThreadedOCPTest-seed#[3C5AC9B5D62D852E]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=4 numCloses=4
[junit4] 2> 1891596 INFO (SUITE-MultiThreadedOCPTest-seed#[3C5AC9B5D62D852E]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 1891597 INFO (SUITE-MultiThreadedOCPTest-seed#[3C5AC9B5D62D852E]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 1891598 INFO (SUITE-MultiThreadedOCPTest-seed#[3C5AC9B5D62D852E]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 1891598 INFO (SUITE-MultiThreadedOCPTest-seed#[3C5AC9B5D62D852E]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 2> 1891600 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1891601 INFO (ZkTestServer Run Thread) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1891601 INFO (ZkTestServer Run Thread) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1891703 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.a.s.c.ZkTestServer start zk server on port:48217
[junit4] 2> 1891703 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.a.s.c.ZkTestServer parse host and port list: localhost:48217
[junit4] 2> 1891703 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.a.s.c.ZkTestServer connecting to localhost 48217
[junit4] 2> 1891710 INFO (zkConnectionManagerCallback-11200-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1891713 INFO (zkConnectionManagerCallback-11202-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1891718 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.a.s.c.ZkTestServer put /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 1891720 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.a.s.c.ZkTestServer put /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 1891721 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.a.s.c.ZkTestServer put /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1891723 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.a.s.c.ZkTestServer put /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 1891724 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.a.s.c.ZkTestServer put /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 1891725 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.a.s.c.ZkTestServer put /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 1891726 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.a.s.c.ZkTestServer put /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 1891727 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.a.s.c.ZkTestServer put /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 1891728 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.a.s.c.ZkTestServer put /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1891729 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.a.s.c.ZkTestServer put /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 1891730 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.a.s.c.ZkTestServer put /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 1891733 INFO (zkConnectionManagerCallback-11206-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1891735 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
[junit4] 2> 1891864 WARN (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.e.j.s.AbstractConnector Ignoring deprecated socket close linger time
[junit4] 2> 1891864 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.a.s.c.s.e.JettySolrRunner Start Jetty (original configured port=0)
[junit4] 2> 1891864 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
[junit4] 2> 1891864 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.e.j.s.Server jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: c4550056e785fb5665914545889f21dc136ad9e6; jvm 1.8.0_191-b12
[junit4] 2> 1891865 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1891865 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1891865 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 1891865 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@47e5d8e2{/,null,AVAILABLE}
[junit4] 2> 1891865 WARN (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@1c00cf0a[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1891867 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@4c00cf{SSL,[ssl, http/1.1]}{127.0.0.1:34629}
[junit4] 2> 1891867 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.e.j.s.Server Started @1893639ms
[junit4] 2> 1891867 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/tempDir-001/control/data, hostContext=/, hostPort=34629, coreRootDirectory=/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/control-001/cores}
[junit4] 2> 1891867 ERROR (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1891867 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 1891867 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.7.0
[junit4] 2> 1891867 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1891867 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1891867 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-12-11T10:29:00.085Z
[junit4] 2> 1891868 INFO (zkConnectionManagerCallback-11208-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1891870 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1891870 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/control-001/solr.xml
[junit4] 2> 1891885 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 1891885 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 1891886 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@57d34b6b, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 1892108 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.a.s.c.ZkContainer Zookeeper client=localhost:48217/solr
[junit4] 2> 1892109 INFO (zkConnectionManagerCallback-11212-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1892112 INFO (zkConnectionManagerCallback-11214-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1892192 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [n:127.0.0.1:34629_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:34629_
[junit4] 2> 1892193 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [n:127.0.0.1:34629_ ] o.a.s.c.Overseer Overseer (id=72210581052588037-127.0.0.1:34629_-n_0000000000) starting
[junit4] 2> 1892199 INFO (zkConnectionManagerCallback-11221-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1892202 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [n:127.0.0.1:34629_ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at localhost:48217/solr ready
[junit4] 2> 1892203 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [n:127.0.0.1:34629_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34629_
[junit4] 2> 1892207 INFO (OverseerStateUpdate-72210581052588037-127.0.0.1:34629_-n_0000000000) [n:127.0.0.1:34629_ ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:34629_
[junit4] 2> 1892208 INFO (OverseerStateUpdate-72210581052588037-127.0.0.1:34629_-n_0000000000) [n:127.0.0.1:34629_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1892211 INFO (OverseerAutoScalingTriggerThread-72210581052588037-127.0.0.1:34629_-n_0000000000) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: []
[junit4] 2> 1892214 INFO (ScheduledTrigger-10807-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 0 and last live nodes: 0
[junit4] 2> 1892216 INFO (zkCallback-11220-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1892223 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [n:127.0.0.1:34629_ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 1892236 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [n:127.0.0.1:34629_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@57d34b6b
[junit4] 2> 1892241 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [n:127.0.0.1:34629_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@57d34b6b
[junit4] 2> 1892242 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [n:127.0.0.1:34629_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@57d34b6b
[junit4] 2> 1892243 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [n:127.0.0.1:34629_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/control-001/cores
[junit4] 2> 1892257 INFO (zkConnectionManagerCallback-11227-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1892258 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1892260 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at localhost:48217/solr ready
[junit4] 2> 1892274 INFO (qtp1461441321-33724) [n:127.0.0.1:34629_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:34629_&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1892278 INFO (OverseerThreadFactory-10809-thread-1-processing-n:127.0.0.1:34629_) [n:127.0.0.1:34629_ ] o.a.s.c.a.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 1892388 INFO (qtp1461441321-33723) [n:127.0.0.1:34629_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1892391 INFO (qtp1461441321-33725) [n:127.0.0.1:34629_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=2
[junit4] 2> 1892392 INFO (qtp1461441321-33726) [n:127.0.0.1:34629_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1892393 INFO (qtp1461441321-33722) [n:127.0.0.1:34629_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1892394 INFO (qtp1461441321-33721) [n:127.0.0.1:34629_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1892397 INFO (qtp1461441321-33723) [n:127.0.0.1:34629_ x:control_collection_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1892397 INFO (qtp1461441321-33723) [n:127.0.0.1:34629_ x:control_collection_shard1_replica_n1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 1893224 INFO (ScheduledTrigger-10807-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1 and last live nodes: 0
[junit4] 2> 1893424 INFO (qtp1461441321-33723) [n:127.0.0.1:34629_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.7.0
[junit4] 2> 1893434 INFO (qtp1461441321-33723) [n:127.0.0.1:34629_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
[junit4] 2> 1893896 INFO (qtp1461441321-33723) [n:127.0.0.1:34629_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 1893919 INFO (qtp1461441321-33723) [n:127.0.0.1:34629_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
[junit4] 2> 1893920 INFO (qtp1461441321-33723) [n:127.0.0.1:34629_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@57d34b6b
[junit4] 2> 1893920 INFO (qtp1461441321-33723) [n:127.0.0.1:34629_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/control-001/cores/control_collection_shard1_replica_n1/data/]
[junit4] 2> 1893921 INFO (qtp1461441321-33723) [n:127.0.0.1:34629_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@65de66b5
[junit4] 2> 1893923 WARN (qtp1461441321-33723) [n:127.0.0.1:34629_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1893969 INFO (qtp1461441321-33723) [n:127.0.0.1:34629_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1893969 INFO (qtp1461441321-33723) [n:127.0.0.1:34629_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1893971 INFO (qtp1461441321-33723) [n:127.0.0.1:34629_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1893971 INFO (qtp1461441321-33723) [n:127.0.0.1:34629_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1893972 INFO (qtp1461441321-33723) [n:127.0.0.1:34629_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=29, maxMergeAtOnceExplicit=20, maxMergedSegmentMB=74.759765625, floorSegmentMB=1.9189453125, forceMergeDeletesPctAllowed=24.5937728965898, segmentsPerTier=13.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.12210359616822314, deletesPctAllowed=23.034847242467393
[junit4] 2> 1893972 INFO (qtp1461441321-33723) [n:127.0.0.1:34629_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5d84d8e8[control_collection_shard1_replica_n1] main]
[junit4] 2> 1893974 INFO (qtp1461441321-33723) [n:127.0.0.1:34629_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1893975 INFO (qtp1461441321-33723) [n:127.0.0.1:34629_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1893975 INFO (qtp1461441321-33723) [n:127.0.0.1:34629_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 1893975 INFO (qtp1461441321-33723) [n:127.0.0.1:34629_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1619550946924167168
[junit4] 2> 1893979 INFO (searcherExecutor-10814-thread-1-processing-n:127.0.0.1:34629_ x:control_collection_shard1_replica_n1 c:control_collection s:shard1) [n:127.0.0.1:34629_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@5d84d8e8[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1893984 INFO (qtp1461441321-33723) [n:127.0.0.1:34629_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/control_collection/terms/shard1 to Terms{values={core_node2=0}, version=0}
[junit4] 2> 1893984 INFO (qtp1461441321-33723) [n:127.0.0.1:34629_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/control_collection/leaders/shard1
[junit4] 2> 1893989 INFO (qtp1461441321-33723) [n:127.0.0.1:34629_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1893989 INFO (qtp1461441321-33723) [n:127.0.0.1:34629_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1893989 INFO (qtp1461441321-33723) [n:127.0.0.1:34629_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:34629/control_collection_shard1_replica_n1/
[junit4] 2> 1893989 INFO (qtp1461441321-33723) [n:127.0.0.1:34629_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1893990 INFO (qtp1461441321-33723) [n:127.0.0.1:34629_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy https://127.0.0.1:34629/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 1893990 INFO (qtp1461441321-33723) [n:127.0.0.1:34629_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 1893990 INFO (qtp1461441321-33723) [n:127.0.0.1:34629_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/control_collection/leaders/shard1/leader after winning as /collections/control_collection/leader_elect/shard1/election/72210581052588037-core_node2-n_0000000000
[junit4] 2> 1893993 INFO (qtp1461441321-33723) [n:127.0.0.1:34629_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:34629/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 1894095 INFO (zkCallback-11213-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1894096 INFO (zkCallback-11213-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1894097 INFO (qtp1461441321-33723) [n:127.0.0.1:34629_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1894100 INFO (qtp1461441321-33723) [n:127.0.0.1:34629_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1704
[junit4] 2> 1894103 INFO (qtp1461441321-33724) [n:127.0.0.1:34629_ ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas
[junit4] 2> 1894201 INFO (zkCallback-11213-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1894201 INFO (zkCallback-11213-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1894201 INFO (zkCallback-11213-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1894202 INFO (qtp1461441321-33724) [n:127.0.0.1:34629_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:34629_&wt=javabin&version=2} status=0 QTime=1928
[junit4] 2> 1894208 INFO (zkConnectionManagerCallback-11232-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1894209 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1894211 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at localhost:48217/solr ready
[junit4] 2> 1894211 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 1894214 INFO (qtp1461441321-33723) [n:127.0.0.1:34629_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1894218 INFO (OverseerThreadFactory-10809-thread-2-processing-n:127.0.0.1:34629_) [n:127.0.0.1:34629_ ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 1894218 INFO (OverseerCollectionConfigSetProcessor-72210581052588037-127.0.0.1:34629_-n_0000000000) [n:127.0.0.1:34629_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1894227 INFO (ScheduledTrigger-10807-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1 and last live nodes: 1
[junit4] 2> 1894424 WARN (OverseerThreadFactory-10809-thread-2-processing-n:127.0.0.1:34629_) [n:127.0.0.1:34629_ ] o.a.s.c.a.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
[junit4] 2> 1894427 INFO (qtp1461441321-33723) [n:127.0.0.1:34629_ ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas
[junit4] 2> 1894427 INFO (qtp1461441321-33723) [n:127.0.0.1:34629_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=2&wt=javabin&version=2} status=0 QTime=213
[junit4] 2> 1894428 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Creating jetty instances pullReplicaCount=0 numOtherReplicas=3
[junit4] 2> 1894563 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/shard-1-001 of type NRT
[junit4] 2> 1894565 WARN (closeThreadPool-11233-thread-1) [ ] o.e.j.s.AbstractConnector Ignoring deprecated socket close linger time
[junit4] 2> 1894565 INFO (closeThreadPool-11233-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Start Jetty (original configured port=0)
[junit4] 2> 1894565 INFO (closeThreadPool-11233-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
[junit4] 2> 1894565 INFO (closeThreadPool-11233-thread-1) [ ] o.e.j.s.Server jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: c4550056e785fb5665914545889f21dc136ad9e6; jvm 1.8.0_191-b12
[junit4] 2> 1894567 INFO (closeThreadPool-11233-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1894567 INFO (closeThreadPool-11233-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1894567 INFO (closeThreadPool-11233-thread-1) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 1894567 INFO (closeThreadPool-11233-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7fcf9b3{/,null,AVAILABLE}
[junit4] 2> 1894567 WARN (closeThreadPool-11233-thread-1) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@6a4828b3[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1894569 INFO (closeThreadPool-11233-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2e572a0f{SSL,[ssl, http/1.1]}{127.0.0.1:56235}
[junit4] 2> 1894569 INFO (closeThreadPool-11233-thread-1) [ ] o.e.j.s.Server Started @1896343ms
[junit4] 2> 1894569 INFO (closeThreadPool-11233-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=56235, coreRootDirectory=/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/shard-1-001/cores}
[junit4] 2> 1894569 ERROR (closeThreadPool-11233-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1894569 INFO (closeThreadPool-11233-thread-1) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 1894569 INFO (closeThreadPool-11233-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.7.0
[junit4] 2> 1894569 INFO (closeThreadPool-11233-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1894569 INFO (closeThreadPool-11233-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1894569 INFO (closeThreadPool-11233-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-12-11T10:29:02.787Z
[junit4] 2> 1894576 INFO (zkConnectionManagerCallback-11235-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1894577 INFO (closeThreadPool-11233-thread-1) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1894577 INFO (closeThreadPool-11233-thread-1) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/shard-1-001/solr.xml
[junit4] 2> 1894580 INFO (closeThreadPool-11233-thread-1) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 1894580 INFO (closeThreadPool-11233-thread-1) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 1894581 INFO (closeThreadPool-11233-thread-1) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@57d34b6b, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 1894735 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/shard-2-001 of type NRT
[junit4] 2> 1894737 WARN (closeThreadPool-11233-thread-2) [ ] o.e.j.s.AbstractConnector Ignoring deprecated socket close linger time
[junit4] 2> 1894737 INFO (closeThreadPool-11233-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Start Jetty (original configured port=0)
[junit4] 2> 1894737 INFO (closeThreadPool-11233-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
[junit4] 2> 1894737 INFO (closeThreadPool-11233-thread-2) [ ] o.e.j.s.Server jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: c4550056e785fb5665914545889f21dc136ad9e6; jvm 1.8.0_191-b12
[junit4] 2> 1894738 INFO (closeThreadPool-11233-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1894738 INFO (closeThreadPool-11233-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1894738 INFO (closeThreadPool-11233-thread-2) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 1894738 INFO (closeThreadPool-11233-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@14b9900a{/,null,AVAILABLE}
[junit4] 2> 1894739 WARN (closeThreadPool-11233-thread-2) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@47bbb4bc[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1894741 INFO (closeThreadPool-11233-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@6356379d{SSL,[ssl, http/1.1]}{127.0.0.1:43090}
[junit4] 2> 1894741 INFO (closeThreadPool-11233-thread-2) [ ] o.e.j.s.Server Started @1896515ms
[junit4] 2> 1894741 INFO (closeThreadPool-11233-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/tempDir-001/jetty2, solrconfig=solrconfig.xml, hostContext=/, hostPort=43090, coreRootDirectory=/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/shard-2-001/cores}
[junit4] 2> 1894741 ERROR (closeThreadPool-11233-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1894741 INFO (closeThreadPool-11233-thread-2) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 1894741 INFO (closeThreadPool-11233-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.7.0
[junit4] 2> 1894741 INFO (closeThreadPool-11233-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1894741 INFO (closeThreadPool-11233-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1894741 INFO (closeThreadPool-11233-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-12-11T10:29:02.959Z
[junit4] 2> 1894743 INFO (zkConnectionManagerCallback-11237-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1894744 INFO (closeThreadPool-11233-thread-2) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1894744 INFO (closeThreadPool-11233-thread-2) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/shard-2-001/solr.xml
[junit4] 2> 1894758 INFO (closeThreadPool-11233-thread-2) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 1894758 INFO (closeThreadPool-11233-thread-2) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 1894759 INFO (closeThreadPool-11233-thread-2) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@57d34b6b, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 1894887 INFO (TEST-MultiThreadedOCPTest.test-seed#[3C5AC9B5D62D852E]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/shard-3-001 of type NRT
[junit4] 2> 1894888 WARN (closeThreadPool-11233-thread-3) [ ] o.e.j.s.AbstractConnector Ignoring deprecated socket close linger time
[junit4] 2> 1894889 INFO (closeThreadPool-11233-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Start Jetty (original configured port=0)
[junit4] 2> 1894889 INFO (closeThreadPool-11233-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
[junit4] 2> 1894889 INFO (closeThreadPool-11233-thread-3) [ ] o.e.j.s.Server jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: c4550056e785fb5665914545889f21dc136ad9e6; jvm 1.8.0_191-b12
[junit4] 2> 1894889 INFO (closeThreadPool-11233-thread-3) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1894889 INFO (closeThreadPool-11233-thread-3) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1894889 INFO (closeThreadPool-11233-thread-3) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 1894890 INFO (closeThreadPool-11233-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@487abad0{/,null,AVAILABLE}
[junit4] 2> 1894890 WARN (closeThreadPool-11233-thread-3) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for SslContextFactory@49eb70fa[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 1894902 INFO (closeThreadPool-11233-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@4783da7{SSL,[ssl, http/1.1]}{127.0.0.1:36150}
[junit4] 2> 1894902 INFO (closeThreadPool-11233-thread-3) [ ] o.e.j.s.Server Started @1896676ms
[junit4] 2> 1894902 INFO (closeThreadPool-11233-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/tempDir-001/jetty3, solrconfig=solrconfig.xml, hostContext=/, hostPort=36150, coreRootDirectory=/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/shard-3-001/cores}
[junit4] 2> 1894902 ERROR (closeThreadPool-11233-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1894902 INFO (closeThreadPool-11233-thread-3) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 1894902 INFO (closeThreadPool-11233-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.7.0
[junit4] 2> 1894903 INFO (closeThreadPool-11233-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1894907 INFO (closeThreadPool-11233-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1894907 INFO (closeThreadPool-11233-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-12-11T10:29:03.125Z
[junit4] 2> 1894909 INFO (zkConnectionManagerCallback-11239-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1894910 INFO (closeThreadPool-11233-thread-3) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1894910 INFO (closeThreadPool-11233-thread-3) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/shard-3-001/solr.xml
[junit4] 2> 1894913 INFO (closeThreadPool-11233-thread-3) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 1894913 INFO (closeThreadPool-11233-thread-3) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 1894914 INFO (closeThreadPool-11233-thread-3) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@57d34b6b, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 1895227 INFO (ScheduledTrigger-10807-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1 and last live nodes: 1
[junit4] 2> 1895243 INFO (closeThreadPool-11233-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=localhost:48217/solr
[junit4] 2> 1895245 INFO (zkConnectionManagerCallback-11243-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1895247 INFO (zkConnectionManagerCallback-11245-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1895251 INFO (closeThreadPool-11233-thread-1) [n:127.0.0.1:56235_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1895253 INFO (closeThreadPool-11233-thread-1) [n:127.0.0.1:56235_ ] o.a.s.c.ZkController Publish node=127.0.0.1:56235_ as DOWN
[junit4] 2> 1895254 INFO (closeThreadPool-11233-thread-1) [n:127.0.0.1:56235_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 1895254 INFO (closeThreadPool-11233-thread-1) [n:127.0.0.1:56235_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56235_
[junit4] 2> 1895255 INFO (zkCallback-11213-thread-3) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1895256 INFO (zkCallback-11220-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1895256 INFO (zkCallback-11231-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1895256 INFO (zkCallback-11244-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1895272 INFO (zkConnectionManagerCallback-11252-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1895273 INFO (closeThreadPool-11233-thread-1) [n:127.0.0.1:56235_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1895274 INFO (closeThreadPool-11233-thread-1) [n:127.0.0.1:56235_ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at localhost:48217/solr ready
[junit4] 2> 1895274 INFO (closeThreadPool-11233-thread-1) [n:127.0.0.1:56235_ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 1895287 INFO (closeThreadPool-11233-thread-1) [n:127.0.0.1:56235_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@57d34b6b
[junit4] 2> 1895292 INFO (closeThreadPool-11233-thread-1) [n:127.0.0.1:56235_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@57d34b6b
[junit4] 2> 1895292 INFO (closeThreadPool-11233-thread-1) [n:127.0.0.1:56235_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@57d34b6b
[junit4] 2> 1895293 INFO (closeThreadPool-11233-thread-1) [n:127.0.0.1:56235_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/shard-1-001/cores
[junit4] 2> 1895524 INFO (closeThreadPool-11233-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=localhost:48217/solr
[junit4] 2> 1895526 INFO (zkConnectionManagerCallback-11257-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1895528 INFO (zkConnectionManagerCallback-11259-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1895532 INFO (closeThreadPool-11233-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=localhost:48217/solr
[junit4] 2> 1895536 INFO (zkConnectionManagerCallback-11265-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1895538 INFO (closeThreadPool-11233-thread-3) [n:127.0.0.1:36150_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1895539 INFO (zkConnectionManagerCallback-11267-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1895542 INFO (closeThreadPool-11233-thread-3) [n:127.0.0.1:36150_ ] o.a.s.c.ZkController Publish node=127.0.0.1:36150_ as DOWN
[junit4] 2> 1895544 INFO (closeThreadPool-11233-thread-3) [n:127.0.0.1:36150_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 1895544 INFO (closeThreadPool-11233-thread-3) [n:127.0.0.1:36150_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36150_
[junit4] 2> 1895545 INFO (zkCallback-11213-thread-3) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1895545 INFO (zkCallback-11231-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1895546 INFO (zkCallback-11220-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1895546 INFO (zkCallback-11244-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1895546 INFO (zkCallback-11251-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1895546 INFO (zkCallback-11258-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1895547 INFO (closeThreadPool-11233-thread-2) [n:127.0.0.1:43090_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 1895551 INFO (closeThreadPool-11233-thread-2) [n:127.0.0.1:43090_ ] o.a.s.c.ZkController Publish node=127.0.0.1:43090_ as DOWN
[junit4] 2> 1895552 INFO (closeThreadPool-11233-thread-2) [n:127.0.0.1:43090_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 1895553 INFO (closeThreadPool-11233-thread-2) [n:127.0.0.1:43090_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43090_
[junit4] 2> 1895554 INFO (zkCallback-11213-thread-3) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1895554 INFO (zkCallback-11244-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1895554 INFO (zkCallback-11231-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1895554 INFO (zkCallback-11220-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1895554 INFO (zkCallback-11251-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1895555 INFO (zkCallback-11258-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1895556 INFO (zkCallback-11266-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1895568 INFO (zkConnectionManagerCallback-11274-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1895569 INFO (zkConnectionManagerCallback-11279-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1895570 INFO (closeThreadPool-11233-thread-3) [n:127.0.0.1:36150_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 1895570 INFO (closeThreadPool-11233-thread-2) [n:127.0.0.1:43090_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 1895572 INFO (closeThreadPool-11233-thread-3) [n:127.0.0.1:36150_ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at localhost:48217/solr ready
[junit4] 2> 1895572 INFO (closeThreadPool-11233-thread-2) [n:127.0.0.1:43090_ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at localhost:48217/solr ready
[junit4] 2> 1895572 INFO (closeThreadPool-11233-thread-3) [n:127.0.0.1:36150_ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 1895572 INFO (closeThreadPool-11233-thread-2) [n:127.0.0.1:43090_ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 1895585 WARN (closeThreadPool-11233-thread-2) [n:127.0.0.1:43090_ ] o.a.s.m.r.j.JmxMetricsReporter Unable to register gauge
[junit4] 2> javax.management.InstanceNotFoundException: solr:dom1=node,category=QUERY,scope=httpShardHandler,name=leasedConnections
[junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1095) ~[?:1.8.0_191]
[junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:427) ~[?:1.8.0_191]
[junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415) ~[?:1.8.0_191]
[junit4] 2> at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546) ~[?:1.8.0_191]
[junit4] 2> at org.apache.solr.metrics.reporters.jmx.JmxMetricsReporter$JmxListener.registerMBean(JmxMetricsReporter.java:531) ~[java/:?]
[junit4] 2> at org.apache.solr.metrics.reporters.jmx.JmxMetricsReporter$JmxListener.onGaugeAdded(JmxMetricsReporter.java:569) ~[java/:?]
[junit4] 2> at org.apache.solr.metrics.reporters.jmx.JmxMetricsReporter.lambda$start$0(JmxMetricsReporter.java:742) ~[java/:?]
[junit4] 2> at java.util.HashMap.forEach(HashMap.java:1289) ~[?:1.8.0_191]
[junit4] 2> at org.apache.solr.metrics.reporters.jmx.JmxMetricsReporter.start(JmxMetricsReporter.java:732) ~[java/:?]
[junit4] 2> at org.apache.solr.metrics.reporters.SolrJmxReporter.doInit(SolrJmxReporter.java:110) ~[java/:?]
[junit4] 2> at org.apache.solr.metrics.SolrMetricReporter.init(SolrMetricReporter.java:70) ~[java/:?]
[junit4] 2> at org.apache.solr.metrics.SolrMetricManager.loadReporter(SolrMetricManager.java:941) ~[java/:?]
[junit4] 2> at org.apache.solr.metrics.SolrMetricManager.loadReporters(SolrMetricManager.java:868) ~[java/:?]
[junit4] 2> at org.apache.solr.core.CoreContainer.load(CoreContainer.java:594) ~[java/:?]
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:253) ~[java/:?]
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:173) ~[java/:?]
[junit4] 2> at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:136) ~[jetty-servlet-9.4.14.v20181114.jar:9.4.14.v20181114]
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.lambda$initialize$0(ServletHandler.java:750) ~[jetty-servlet-9.4.14.v20181114.jar:9.4.14.v20181114]
[junit4] 2> at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948) ~[?:1.8.0_191]
[junit4] 2> at java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:742) ~[?:1.8.0_191]
[junit4] 2> at java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:742) ~[?:1.8.0_191]
[junit4] 2> at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580) ~[?:1.8.0_191]
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:744) ~[jetty-servlet-9.4.14.v20181114.jar:9.4.14.v20181114]
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1449) ~[jetty-servlet-9.4.14.v20181114.jar:9.4.14.v20181114]
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1513) ~[jetty-servlet-9.4.14.v20181114.jar:9.4.14.v20181114]
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1158) ~[jetty-servlet-9.4.14.v20181114.jar:9.4.14.v20181114]
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:995) ~[jetty-servlet-9.4.14.v20181114.jar:9.4.14.v20181114]
[junit4] 2> at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:467) ~[jetty-servlet-9.4.14.v20181114.jar:9.4.14.v20181114]
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:347) ~[java/:?]
[junit4] 2> at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179) ~[jetty-util-9.4.14.v20181114.jar:9.4.14.v20181114]
[junit4] 2> at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69) ~[jetty-util-9.4.14.v20181114.jar:9.4.14.v20181114]
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner.retryOnPortBindFailure(JettySolrRunner.java:510) ~[java/:?]
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:449) ~[java/:?]
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:417) ~[java/:?]
[junit4] 2> at org.apache.solr.cloud.AbstractFullDistribZkTestBase.lambda$createJettys$2(AbstractFullDistribZkTestBase.java:464) ~[java/:?]
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_191]
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_191]
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) ~[java/:?]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_191]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_191]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191]
[junit4] 2> 1895588 INFO (closeThreadPool-11233-thread-3) [n:127.0.0.1:36150_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@57d34b6b
[junit4] 2> 1895589 INFO (closeThreadPool-11233-thread-2) [n:127.0.0.1:43090_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@57d34b6b
[junit4] 2> 1895594 INFO (closeThreadPool-11233-thread-3) [n:127.0.0.1:36150_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@57d34b6b
[junit4] 2> 1895594 INFO (closeThreadPool-11233-thread-3) [n:127.0.0.1:36150_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@57d34b6b
[junit4] 2> 1895595 INFO (closeThreadPool-11233-thread-3) [n:127.0.0.1:36150_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/shard-3-001/cores
[junit4] 2> 1895595 INFO (closeThreadPool-11233-thread-2) [n:127.0.0.1:43090_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@57d34b6b
[junit4] 2> 1895595 INFO (closeThreadPool-11233-thread-2) [n:127.0.0.1:43090_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@57d34b6b
[junit4] 2> 1895596 INFO (closeThreadPool-11233-thread-2) [n:127.0.0.1:43090_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_3C5AC9B5D62D852E-001/shard-2-001/cores
[junit4] 2> 1895652 INFO (qtp1461441321-33724) [n:127.0.0.1:34629_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:56235_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1895657 INFO (qtp1461441321-33726) [n:127.0.0.1:34629_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:43090_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1895657 INFO (qtp1461441321-33725) [n:127.0.0.1:34629_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:36150_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1895657 INFO (OverseerCollectionConfigSetProcessor-72210581052588037-127.0.0.1:34629_-n_0000000000) [n:127.0.0.1:34629_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1895661 INFO (qtp1461441321-33722) [n:127.0.0.1:34629_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.control_collection.shard1.replica_n1:INDEX.sizeInBytes} status=0 QTime=1
[junit4] 2> 1895661 INFO (qtp1461441321-33721) [n:127.0.0.1:34629_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.control_collection.shard1.replica_n1:INDEX.sizeInBytes} status=0 QTime=0
[junit4] 2> 1895662 INFO (qtp1461441321-33723) [n:127.0.0.1:34629_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.control_collection.shard1.replica_n1:INDEX.sizeInBytes} status=0 QTime=0
[junit4] 2> 1895663 INFO (qtp1461441321-33722) [n:127.0.0.1:34629_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1895663 INFO (qtp1461441321-33721) [n:127.0.0.1:34629_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1895664 INFO (qtp1461441321-33723) [n:127.0.0.1:34629_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1895665 INFO (qtp1461441321-33722) [n:127.0.0.1:34629_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1895666 INFO (qtp1461441321-33721) [n:127.0.0.1:34629_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1895682 INFO (qtp1322148053-33808) [n:127.0.0.1:36150_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1895683 INFO (qtp1322148053-33816) [n:127.0.0.1:36150_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1895684 INFO (qtp1322148053-33813) [n:127.0.0.1:36150_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1895685 INFO (qtp1322148053-33811) [n:127.0.0.1:36150_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1895694 INFO (qtp1322148053-33809) [n:127.0.0.1:36150_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1895712 INFO (qtp2109900352-33796) [n:127.0.0.1:43090_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1895713 INFO (qtp2109900352-33797) [n:127.0.0.1:43090_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1895714 INFO (qtp2109900352-33799) [n:127.0.0.1:43090_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1895714 INFO (qtp2109900352-33798) [n:127.0.0.1:43090_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1895715 INFO (qtp2109900352-33800) [n:127.0.0.1:43090_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1895732 INFO (qtp709852480-33786) [n:127.0.0.1:56235_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1895733 INFO (qtp709852480-33787) [n:127.0.0.1:56235_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1895734 INFO (qtp709852480-33788) [n:127.0.0.1:56235_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1895735 INFO (qtp709852480-33783) [n:127.0.0.1:56235_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1895735 INFO (qtp709852480-33784) [n:127.0.0.1:56235_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1895737 INFO (OverseerThreadFactory-10809-thread-3-processing-n:127.0.0.1:34629_) [n:127.0.0.1:34629_ c:collection1 s:shard2 ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:56235_ for creating new replica of shard shard2 for collection collection1
[junit4] 2> 1895738 INFO (OverseerThreadFactory-10809-thread-4-processing-n:127.0.0.1:34629_) [n:127.0.0.1:34629_ c:collection1 s:shard1 ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:43090_ for creating new replica of shard shard1 for collection collection1
[junit4] 2> 1895740 INFO (OverseerThreadFactory-10809-thread-3-processing-n:127.0.0.1:34629_) [n:127.0.0.1:34629_ c:collection1 s:shard2 ] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
[junit4] 2> 1895741 INFO (OverseerThreadFactory-10809-thread-4-processing-n:127.0.0.1:34629_) [n:127.0.0.1:34629_ c:collection1 s:shard1 ] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
[junit4] 2> 1895746 INFO (qtp2109900352-33796) [n:127.0.0.1:43090_ x:collection1_shard1_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n2&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1895746 INFO (qtp709852480-33787) [n:127.0.0.1:56235_ x:collection1_shard2_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n1&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1896233 INFO (ScheduledTrigger-10807-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4 and last live nodes: 1
[junit4] 2> 1896764 INFO (qtp2109900352-33796) [n:127.0.0.1:43090_ c:collection1 s:shard1 x:collection1_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.7.0
[junit4] 2> 1896765 INFO (qtp709852480-33787) [n:127.0.0.1:56235_ c:collection1 s:shard2 x:collection1_shard2_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.7.0
[junit4] 2> 1896782 INFO (qtp2109900352-33796) [n:127.0.0.1:43090_ c:collection1 s:shard1 x:collection1_shard1_replica_n2] o.a.s.s.IndexSchema [collection1_shard1_replica_n2] Schema name=test
[junit4] 2> 1896782 INFO (qtp709852480-33787) [n:127.0.0.1:56235_ c:collection1 s:shard2 x:collection1_shard2_replica_n1] o.a.s.s.IndexSchema [collection1_shard2_replica_n1] Schema name=test
[junit4] 2> 1896907 INFO (qtp2109900352-33796) [n:127.0.0.1:43090_ c:collection1 s:shard1 x:collection1_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 1896908 INFO (qtp709852480-33787) [n:127.0.0.1:56235_ c:collection1 s:shard2 x:collection1_shard2_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 1896930 INFO (qtp2109900352-33796) [n:127.0.0.1:43090_ c:collection1 s:shard1 x:collection1_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n2' using configuration from collection collection1, trusted=tru
[...truncated too long message...]
nit4] 1> java.lang.InterruptedException: null
[junit4] 1> at java.lang.Object.wait(Native Method) ~[?:1.8.0_191]
[junit4] 1> at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:147) [zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
[junit4] 2> Dec 11, 2018 10:57:05 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> SEVERE: There are still zombie threads that couldn't be terminated:
[junit4] 2> 1) Thread[id=6624, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-SolrZkClientTest]
[junit4] 2> at sun.nio.ch.DevPollArrayWrapper.interrupt(Native Method)
[junit4] 2> at sun.nio.ch.DevPollArrayWrapper.interrupt(DevPollArrayWrapper.java:300)
[junit4] 2> at sun.nio.ch.DevPollSelectorImpl.wakeup(DevPollSelectorImpl.java:207)
[junit4] 2> at java.nio.channels.spi.AbstractSelector$1.interrupt(AbstractSelector.java:213)
[junit4] 2> at java.nio.channels.spi.AbstractSelector.begin(AbstractSelector.java:219)
[junit4] 2> at sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:97)
[junit4] 2> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
[junit4] 2> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:196)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=1186, maxMBSortInHeap=5.053221858295194, sim=RandomSimilarity(queryNorm=false): {}, locale=en-MT, timezone=Europe/Paris
[junit4] 2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_191 (64-bit)/cpus=3,threads=2,free=291442256,total=503316480
[junit4] 2> NOTE: All tests run in this JVM: [SolrQueryTest, AnlysisResponseBaseTest, ExclusiveOrEvaluatorTest, TestValidatingJsonMap, MultiplyEvaluatorTest, StreamingTest, ModifiableSolrParamsTest, CloudSolrClientCacheTest, TestSolrJErrorHandling, TestCollectionStateWatchers, TestSuggesterResponse, SolrParamTest, RangeFacetMapTest, GraphTest, SubtractEvaluatorTest, TestCollectionAdminRequest, HttpClientUtilTest, StreamExpressionToExpessionTest, TestDelegationTokenResponse, SolrExampleJettyTest, JsonQueryRequestUnitTest, TestPathTrie, NormalDistributionEvaluatorTest, LessThanEvaluatorTest, CloudSolrClientRetryTest, TestLang, TestSolrJsonWriter, ReverseEvaluatorTest, TestV1toV2ApiMapper, CloudSolrClientBadInputTest, DirectJsonQueryRequestFacetingIntegrationTest, ArrayEvaluatorTest, CumulativeProbabilityEvaluatorTest, UuidEvaluatorTest, SchemaTest, SolrExampleStreamingTest, ConcurrentUpdateSolrClientBadInputTest, CosineEvaluatorTest, NoOpResponseParserTest, TestJsonRecordReader, HttpSolrClientSSLAuthConPoolTest, SolrExceptionTest, TestBatchUpdate, TestLBHttpSolrClient, ConcurrentUpdateSolrClientMultiCollectionTest, HttpSolrClientBuilderTest, HttpSolrClientConPoolTest, JDBCStreamTest, MathExpressionTest, StreamExpressionTest, AddEvaluatorTest, AndEvaluatorTest, AppendEvaluatorTest, ArcCosineEvaluatorTest, ArcSineEvaluatorTest, ArcTangentEvaluatorTest, CeilingEvaluatorTest, CoalesceEvaluatorTest, EmpiricalDistributionEvaluatorTest, EqualToEvaluatorTest, FloorEvaluatorTest, GreaterThanEqualToEvaluatorTest, GreaterThanEvaluatorTest, HyperbolicCosineEvaluatorTest, HyperbolicSineEvaluatorTest, HyperbolicTangentEvaluatorTest, LengthEvaluatorTest, LessThanEqualToEvaluatorTest, NotEvaluatorTest, OrEvaluatorTest, PowerEvaluatorTest, RawValueEvaluatorTest, RecursiveEvaluatorTest, RegressionEvaluatorTest, SineEvaluatorTest, SquareRootEvaluatorTest, UniformDistributionEvaluatorTest, ConcatOperationTest, OperationsTest, TestCoreAdmin, FacetFieldTest, FieldAnalysisResponseTest, TermsResponseTest, TestClusteringResponse, SolrDocumentTest, TestToleratedUpdateError, SolrZkClientTest]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=SolrZkClientTest -Dtests.seed=C23229FC9129A0C6 -Dtests.slow=true -Dtests.locale=en-MT -Dtests.timezone=Europe/Paris -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J0 | SolrZkClientTest (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 5 threads leaked from SUITE scope at org.apache.solr.common.cloud.SolrZkClientTest:
[junit4] > 1) Thread[id=6626, name=SyncThread:0, state=WAITING, group=TGRP-SolrZkClientTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:127)
[junit4] > 2) Thread[id=6627, name=ProcessThread(sid:0 cport:49726):, state=WAITING, group=TGRP-SolrZkClientTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:123)
[junit4] > 3) Thread[id=6625, name=SessionTracker, state=TIMED_WAITING, group=TGRP-SolrZkClientTest]
[junit4] > at java.lang.Object.wait(Native Method)
[junit4] > at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:147)
[junit4] > 4) Thread[id=6624, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-SolrZkClientTest]
[junit4] > at sun.nio.ch.DevPollArrayWrapper.poll0(Native Method)
[junit4] > at sun.nio.ch.DevPollArrayWrapper.poll(DevPollArrayWrapper.java:223)
[junit4] > at sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:98)
[junit4] > at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
[junit4] > at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
[junit4] > at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:196)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 5) Thread[id=6623, name=ZkTestServer Run Thread, state=WAITING, group=TGRP-SolrZkClientTest]
[junit4] > at java.lang.Object.wait(Native Method)
[junit4] > at java.lang.Thread.join(Thread.java:1252)
[junit4] > at java.lang.Thread.join(Thread.java:1326)
[junit4] > at org.apache.zookeeper.server.NIOServerCnxnFactory.join(NIOServerCnxnFactory.java:313)
[junit4] > at org.apache.solr.cloud.ZkTestServer$ZKServerMain.runFromConfig(ZkTestServer.java:343)
[junit4] > at org.apache.solr.cloud.ZkTestServer$2.run(ZkTestServer.java:564)
[junit4] > at __randomizedtesting.SeedInfo.seed([C23229FC9129A0C6]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=6624, name=NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0, state=RUNNABLE, group=TGRP-SolrZkClientTest]
[junit4] > at sun.nio.ch.DevPollArrayWrapper.interrupt(Native Method)
[junit4] > at sun.nio.ch.DevPollArrayWrapper.interrupt(DevPollArrayWrapper.java:300)
[junit4] > at sun.nio.ch.DevPollSelectorImpl.wakeup(DevPollSelectorImpl.java:207)
[junit4] > at java.nio.channels.spi.AbstractSelector$1.interrupt(AbstractSelector.java:213)
[junit4] > at java.nio.channels.spi.AbstractSelector.begin(AbstractSelector.java:219)
[junit4] > at sun.nio.ch.DevPollSelectorImpl.doSelect(DevPollSelectorImpl.java:97)
[junit4] > at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
[junit4] > at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
[junit4] > at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:196)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([C23229FC9129A0C6]:0)
[junit4] Completed [174/174 (1!)] on J0 in 6.36s, 2 tests, 3 errors <<< FAILURES!
[...truncated 7874 lines...]
BUILD FAILED
/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/build.xml:633: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-7.x-Solaris/build.xml:585: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid24294.hprof
Total time: 86 minutes 29 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2