You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2014/12/19 02:22:35 UTC
[JENKINS] Lucene-Solr-5.x-Linux (32bit/jdk1.8.0_40-ea-b09) - Build
# 11602 - Still Failing!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Linux/11602/
Java: 32bit/jdk1.8.0_40-ea-b09 -client -XX:+UseG1GC (asserts: true)
4 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest
Error Message:
ERROR: SolrZkClient opens=21 closes=20
Stack Trace:
java.lang.AssertionError: ERROR: SolrZkClient opens=21 closes=20
at __randomizedtesting.SeedInfo.seed([7269034E4F253D70]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.SolrTestCaseJ4.endTrackingZkClients(SolrTestCaseJ4.java:461)
at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:188)
at sun.reflect.GeneratedMethodAccessor52.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:790)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest
Error Message:
6 threads leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest: 1) Thread[id=5706, name=zkCallback-751-thread-4, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 2) Thread[id=5608, name=TEST-CollectionsAPIDistributedZkTest.testDistribSearch-seed#[7269034E4F253D70]-SendThread(127.0.0.1:54370), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003) 3) Thread[id=5685, name=zkCallback-751-thread-3, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 4) Thread[id=5678, name=zkCallback-751-thread-2, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 5) Thread[id=5609, name=TEST-CollectionsAPIDistributedZkTest.testDistribSearch-seed#[7269034E4F253D70]-EventThread, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494) 6) Thread[id=5610, name=zkCallback-751-thread-1, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 6 threads leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest:
1) Thread[id=5706, name=zkCallback-751-thread-4, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2) Thread[id=5608, name=TEST-CollectionsAPIDistributedZkTest.testDistribSearch-seed#[7269034E4F253D70]-SendThread(127.0.0.1:54370), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
3) Thread[id=5685, name=zkCallback-751-thread-3, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
4) Thread[id=5678, name=zkCallback-751-thread-2, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
5) Thread[id=5609, name=TEST-CollectionsAPIDistributedZkTest.testDistribSearch-seed#[7269034E4F253D70]-EventThread, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
6) Thread[id=5610, name=zkCallback-751-thread-1, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([7269034E4F253D70]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=5706, name=zkCallback-751-thread-4, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 2) Thread[id=5608, name=TEST-CollectionsAPIDistributedZkTest.testDistribSearch-seed#[7269034E4F253D70]-SendThread(127.0.0.1:54370), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003) 3) Thread[id=5685, name=zkCallback-751-thread-3, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 4) Thread[id=5678, name=zkCallback-751-thread-2, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 5) Thread[id=5610, name=zkCallback-751-thread-1, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=5706, name=zkCallback-751-thread-4, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2) Thread[id=5608, name=TEST-CollectionsAPIDistributedZkTest.testDistribSearch-seed#[7269034E4F253D70]-SendThread(127.0.0.1:54370), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
3) Thread[id=5685, name=zkCallback-751-thread-3, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
4) Thread[id=5678, name=zkCallback-751-thread-2, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
5) Thread[id=5610, name=zkCallback-751-thread-1, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([7269034E4F253D70]:0)
FAILED: org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testDistribSearch
Error Message:
expected:<...9034E4F253D70-001/te[mpDir-003/solrj_test_core_props_shard1_replica1]> but was:<...9034E4F253D70-001/te[stPropertyParamsForCreate-001/instanceDir-fxy]>
Stack Trace:
org.junit.ComparisonFailure: expected:<...9034E4F253D70-001/te[mpDir-003/solrj_test_core_props_shard1_replica1]> but was:<...9034E4F253D70-001/te[stPropertyParamsForCreate-001/instanceDir-fxy]>
at __randomizedtesting.SeedInfo.seed([7269034E4F253D70:F38F8D56387A5D4C]:0)
at org.junit.Assert.assertEquals(Assert.java:125)
at org.junit.Assert.assertEquals(Assert.java:147)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.checkInstanceDirs(CollectionsAPIDistributedZkTest.java:1154)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:901)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:205)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869)
at sun.reflect.GeneratedMethodAccessor68.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 9811 lines...]
[junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-7269034E4F253D70-001/init-core-data-001
[junit4] 2> 2128339 T5417 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false)
[junit4] 2> 2128339 T5417 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4] 2> 2128344 T5417 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> 2128344 T5417 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2128345 T5418 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 2128445 T5417 oasc.ZkTestServer.run start zk server on port:54370
[junit4] 2> 2128446 T5417 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 2128447 T5417 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2128449 T5425 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@55d4ea name:ZooKeeperConnection Watcher:127.0.0.1:54370 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2128449 T5417 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2128450 T5417 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 2128450 T5417 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 2128453 T5417 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 2128454 T5417 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2128455 T5428 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@15fafbd name:ZooKeeperConnection Watcher:127.0.0.1:54370/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2128455 T5417 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2128455 T5417 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 2128456 T5417 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 2128458 T5417 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 2128460 T5417 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 2128461 T5417 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 2128464 T5417 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 2128464 T5417 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 2128467 T5417 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 2128468 T5417 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 2128470 T5417 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 2128471 T5417 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 2128472 T5417 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 2128473 T5417 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 2128475 T5417 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 2128475 T5417 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 2128477 T5417 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 2128478 T5417 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 2128480 T5417 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 2128480 T5417 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 2128482 T5417 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 2128482 T5417 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 2128484 T5417 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 2128485 T5417 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 2128486 T5417 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 2128487 T5417 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 2128488 T5417 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 2128489 T5417 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 2128491 T5417 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 2128492 T5417 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2128493 T5431 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3d460c name:ZooKeeperConnection Watcher:127.0.0.1:54370/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2128493 T5417 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2128494 T5417 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> Using old style solr.xml
[junit4] 2> 2128495 T5417 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 2128496 T5417 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2128497 T5434 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@17086be name:ZooKeeperConnection Watcher:127.0.0.1:54370 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2128497 T5417 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2128498 T5417 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 2128498 T5417 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 2128500 T5417 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 2128500 T5417 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2128501 T5437 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1878f6c name:ZooKeeperConnection Watcher:127.0.0.1:54370/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2128502 T5417 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2128502 T5417 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 2128503 T5417 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf2/solrconfig.xml
[junit4] 2> 2128503 T5417 oascc.SolrZkClient.makePath makePath: /configs/conf2/solrconfig.xml
[junit4] 2> 2128505 T5417 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf2/schema.xml
[junit4] 2> 2128506 T5417 oascc.SolrZkClient.makePath makePath: /configs/conf2/schema.xml
[junit4] 2> 2128508 T5417 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf2/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 2128508 T5417 oascc.SolrZkClient.makePath makePath: /configs/conf2/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 2128510 T5417 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf2/stopwords.txt
[junit4] 2> 2128510 T5417 oascc.SolrZkClient.makePath makePath: /configs/conf2/stopwords.txt
[junit4] 2> 2128514 T5417 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf2/protwords.txt
[junit4] 2> 2128515 T5417 oascc.SolrZkClient.makePath makePath: /configs/conf2/protwords.txt
[junit4] 2> 2128517 T5417 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf2/currency.xml
[junit4] 2> 2128517 T5417 oascc.SolrZkClient.makePath makePath: /configs/conf2/currency.xml
[junit4] 2> 2128519 T5417 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf2/open-exchange-rates.json
[junit4] 2> 2128520 T5417 oascc.SolrZkClient.makePath makePath: /configs/conf2/open-exchange-rates.json
[junit4] 2> 2128521 T5417 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf2/mapping-ISOLatin1Accent.txt
[junit4] 2> 2128522 T5417 oascc.SolrZkClient.makePath makePath: /configs/conf2/mapping-ISOLatin1Accent.txt
[junit4] 2> 2128524 T5417 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf2/old_synonyms.txt
[junit4] 2> 2128524 T5417 oascc.SolrZkClient.makePath makePath: /configs/conf2/old_synonyms.txt
[junit4] 2> 2128526 T5417 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf2/synonyms.txt
[junit4] 2> 2128527 T5417 oascc.SolrZkClient.makePath makePath: /configs/conf2/synonyms.txt
[junit4] 2> 2128528 T5417 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/elevate.xml to /configs/conf2/elevate.xml
[junit4] 2> 2128529 T5417 oascc.SolrZkClient.makePath makePath: /configs/conf2/elevate.xml
[junit4] 2> 2128830 T5417 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 2128835 T5417 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 2128845 T5417 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:56216
[junit4] 2> 2128847 T5417 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 2128848 T5417 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 2128848 T5417 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-7269034E4F253D70-001/tempDir-002
[junit4] 2> 2128849 T5417 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-7269034E4F253D70-001/tempDir-002/'
[junit4] 2> 2128870 T5417 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-7269034E4F253D70-001/tempDir-002/solr.xml
[junit4] 2> 2128893 T5417 oasc.CoreContainer.<init> New CoreContainer 1202061
[junit4] 2> 2128893 T5417 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-7269034E4F253D70-001/tempDir-002/]
[junit4] 2> 2128894 T5417 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 2128895 T5417 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
[junit4] 2> 2128895 T5417 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 2128895 T5417 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 2128896 T5417 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
[junit4] 2> 2128896 T5417 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 2128897 T5417 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 2128897 T5417 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 2128897 T5417 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 2128898 T5417 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 2128899 T5417 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=3400000&connTimeout=45000&retry=false
[junit4] 2> 2128900 T5417 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 2128901 T5417 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 2128901 T5417 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 2128902 T5417 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:54370/solr
[junit4] 2> 2128902 T5417 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 2128902 T5417 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 2128903 T5417 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2128904 T5449 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1119ff6 name:ZooKeeperConnection Watcher:127.0.0.1:54370 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2128905 T5417 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2128906 T5417 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 2128907 T5417 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2128908 T5452 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@15fefc6 name:ZooKeeperConnection Watcher:127.0.0.1:54370/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2128909 T5417 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2128910 T5417 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 2128912 T5417 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 2128915 T5417 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 2128917 T5417 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 2128919 T5417 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 2128921 T5417 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 2128922 T5417 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56216_
[junit4] 2> 2128923 T5417 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:56216_
[junit4] 2> 2128925 T5417 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 2128926 T5417 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 2128928 T5417 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 2128929 T5417 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:56216_
[junit4] 2> 2128930 T5417 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 2128931 T5417 oasc.Overseer.start Overseer (id=92992433486036998-127.0.0.1:56216_-n_0000000000) starting
[junit4] 2> 2128933 T5417 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 2128939 T5417 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 2128940 T5454 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 2128941 T5417 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 2128943 T5417 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 2128945 T5417 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2128948 T5453 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 2128951 T5417 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1
[junit4] 2> 2128951 T5417 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 2128955 T5417 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 2128956 T5417 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2128957 T5459 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@ddf01f name:ZooKeeperConnection Watcher:127.0.0.1:54370/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2128958 T5417 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2128958 T5417 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 2128959 T5417 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2128962 T5417 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 2129338 T5417 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 2129339 T5417 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 2129342 T5417 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 2129344 T5417 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:42440
[junit4] 2> 2129346 T5417 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 2129347 T5417 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 2129347 T5417 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-7269034E4F253D70-001/tempDir-003
[junit4] 2> 2129348 T5417 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-7269034E4F253D70-001/tempDir-003/'
[junit4] 2> 2129370 T5417 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-7269034E4F253D70-001/tempDir-003/solr.xml
[junit4] 2> 2129390 T5417 oasc.CoreContainer.<init> New CoreContainer 17117486
[junit4] 2> 2129391 T5417 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-7269034E4F253D70-001/tempDir-003/]
[junit4] 2> 2129392 T5417 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 2129392 T5417 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
[junit4] 2> 2129393 T5417 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 2129393 T5417 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 2129394 T5417 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
[junit4] 2> 2129394 T5417 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 2129395 T5417 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 2129395 T5417 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 2129395 T5417 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 2129396 T5417 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 2129398 T5417 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=3400000&connTimeout=45000&retry=false
[junit4] 2> 2129399 T5417 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 2129399 T5417 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 2129400 T5417 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 2129400 T5417 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:54370/solr
[junit4] 2> 2129400 T5417 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 2129401 T5417 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 2129402 T5417 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2129403 T5471 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@402f00 name:ZooKeeperConnection Watcher:127.0.0.1:54370 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2129404 T5417 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2129404 T5417 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 2129407 T5417 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2129408 T5474 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@ab93ba name:ZooKeeperConnection Watcher:127.0.0.1:54370/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2129408 T5417 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2129416 T5417 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2130420 T5417 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42440_
[junit4] 2> 2130421 T5417 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:42440_
[junit4] 2> 2130426 T5417 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 2130429 T5417 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1
[junit4] 2> 2130430 T5417 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 2130834 T5417 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 2130835 T5417 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 2130840 T5417 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 2130842 T5417 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:47176
[junit4] 2> 2130844 T5417 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 2130845 T5417 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 2130846 T5417 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-7269034E4F253D70-001/tempDir-004
[junit4] 2> 2130846 T5417 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-7269034E4F253D70-001/tempDir-004/'
[junit4] 2> 2130873 T5417 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-7269034E4F253D70-001/tempDir-004/solr.xml
[junit4] 2> 2130894 T5417 oasc.CoreContainer.<init> New CoreContainer 5759407
[junit4] 2> 2130895 T5417 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-7269034E4F253D70-001/tempDir-004/]
[junit4] 2> 2130895 T5417 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 2130896 T5417 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
[junit4] 2> 2130897 T5417 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 2130897 T5417 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 2130898 T5417 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
[junit4] 2> 2130898 T5417 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 2130898 T5417 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 2130899 T5417 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 2130899 T5417 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 2130900 T5417 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 2130902 T5417 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=3400000&connTimeout=45000&retry=false
[junit4] 2> 2130904 T5417 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 2130905 T5417 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 2130905 T5417 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 2130906 T5417 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:54370/solr
[junit4] 2> 2130906 T5417 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 2130907 T5417 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 2130908 T5417 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2130909 T5487 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@bd9716 name:ZooKeeperConnection Watcher:127.0.0.1:54370 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2130910 T5417 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2130911 T5417 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 2130913 T5417 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2130914 T5490 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@874779 name:ZooKeeperConnection Watcher:127.0.0.1:54370/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2130914 T5417 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2130919 T5417 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2131923 T5417 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47176_
[junit4] 2> 2131924 T5417 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:47176_
[junit4] 2> 2131927 T5417 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 2131930 T5417 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1
[junit4] 2> 2131931 T5417 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 2132201 T5417 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
[junit4] 2> 2132201 T5417 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 2132204 T5417 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 2132206 T5417 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:54524
[junit4] 2> 2132208 T5417 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 2132208 T5417 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 2132208 T5417 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-7269034E4F253D70-001/tempDir-005
[junit4] 2> 2132209 T5417 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-7269034E4F253D70-001/tempDir-005/'
[junit4] 2> 2132226 T5417 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-7269034E4F253D70-001/tempDir-005/solr.xml
[junit4] 2> 2132242 T5417 oasc.CoreContainer.<init> New CoreContainer 12233286
[junit4] 2> 2132242 T5417 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-7269034E4F253D70-001/tempDir-005/]
[junit4] 2> 2132243 T5417 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 2132243 T5417 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
[junit4] 2> 2132244 T5417 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 2132244 T5417 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 2132245 T5417 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
[junit4] 2> 2132245 T5417 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 2132245 T5417 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 2132246 T5417 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 2132246 T5417 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 2132246 T5417 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 2132248 T5417 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=3400000&connTimeout=45000&retry=false
[junit4] 2> 2132249 T5417 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 2132249 T5417 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 2132249 T5417 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 2132250 T5417 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:54370/solr
[junit4] 2> 2132250 T5417 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 2132251 T5417 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 2132251 T5417 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2132252 T5503 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@cfe470 name:ZooKeeperConnection Watcher:127.0.0.1:54370 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2132253 T5417 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2132254 T5417 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 2132256 T5417 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2132257 T5506 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@113cb46 name:ZooKeeperConnection Watcher:127.0.0.1:54370/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2132257 T5417 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2132263 T5417 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2133268 T5417 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54524_
[junit4] 2> 2133269 T5417 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:54524_
[junit4] 2> 2133274 T5417 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 2133279 T5417 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1
[junit4] 2> 2133280 T5417 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 2133572 T5417 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 4
[junit4] 2> 2133573 T5417 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 2133576 T5417 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 2133577 T5417 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:54667
[junit4] 2> 2133579 T5417 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 2133579 T5417 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 2133580 T5417 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-7269034E4F253D70-001/tempDir-006
[junit4] 2> 2133580 T5417 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-7269034E4F253D70-001/tempDir-006/'
[junit4] 2> 2133597 T5417 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-7269034E4F253D70-001/tempDir-006/solr.xml
[junit4] 2> 2133612 T5417 oasc.CoreContainer.<init> New CoreContainer 10029646
[junit4] 2> 2133612 T5417 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-7269034E4F253D70-001/tempDir-006/]
[junit4] 2> 2133613 T5417 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 2133614 T5417 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
[junit4] 2> 2133614 T5417 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 2133614 T5417 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 2133615 T5417 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
[junit4] 2> 2133615 T5417 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 2133615 T5417 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 2133616 T5417 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 2133616 T5417 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 2133616 T5417 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 2133618 T5417 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=3400000&connTimeout=45000&retry=false
[junit4] 2> 2133619 T5417 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 2133619 T5417 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 2133620 T5417 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 2133620 T5417 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:54370/solr
[junit4] 2> 2133621 T5417 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 2133621 T5417 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 2133622 T5417 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2133623 T5519 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@114d926 name:ZooKeeperConnection Watcher:127.0.0.1:54370 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2133624 T5417 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2133624 T5417 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 2133626 T5417 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2133626 T5522 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f6859b name:ZooKeeperConnection Watcher:127.0.0.1:54370/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2133627 T5417 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2133631 T5417 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2134634 T5417 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54667_
[junit4] 2> 2134635 T5417 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:54667_
[junit4] 2> 2134639 T5417 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 2134643 T5417 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1
[junit4] 2> 2134644 T5417 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 2134728 T5465 oasha.CollectionsHandler.handleCreateAction Creating Collection : replicationFactor=2&collection.configName=conf1&router.field=myOwnField&name=solrj_collection&action=CREATE&numShards=2&wt=javabin&version=2
[junit4] 2> 2134729 T5452 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/collection-queue-work state SyncConnected
[junit4] 2> 2134731 T5454 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
[junit4] 2> "operation":"create",
[junit4] 2> "fromApi":"true",
[junit4] 2> "name":"solrj_collection",
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "collection.configName":"conf1",
[junit4] 2> "numShards":"2",
[junit4] 2> "router.field":"myOwnField"}
[junit4] 2> 2134732 T5524 oasc.OverseerCollectionProcessor.processMessage WARN OverseerCollectionProcessor.processMessage : create , {
[junit4] 2> "operation":"create",
[junit4] 2> "fromApi":"true",
[junit4] 2> "name":"solrj_collection",
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "collection.configName":"conf1",
[junit4] 2> "numShards":"2",
[junit4] 2> "router.field":"myOwnField"}
[junit4] 2> 2134733 T5524 oasc.OverseerCollectionProcessor.createConfNode creating collections conf node /collections/solrj_collection
[junit4] 2> 2134734 T5524 oascc.SolrZkClient.makePath makePath: /collections/solrj_collection
[junit4] 2> 2134736 T5452 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 2134739 T5453 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 0, message = {
[junit4] 2> "operation":"create",
[junit4] 2> "fromApi":"true",
[junit4] 2> "name":"solrj_collection",
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "collection.configName":"conf1",
[junit4] 2> "numShards":"2",
[junit4] 2> "router.field":"myOwnField"} current state version: 0
[junit4] 2> 2134739 T5453 oasco.ClusterStateMutator.createCollection building a new cName: solrj_collection
[junit4] 2> 2134740 T5474 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 2134741 T5522 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 2134741 T5490 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 2134741 T5452 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 2134741 T5459 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 2134741 T5506 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 2134937 T5524 oasc.OverseerCollectionProcessor.createCollection Creating SolrCores for new collection solrj_collection, shardNames [shard1, shard2] , replicationFactor : 2
[junit4] 2> 2134937 T5524 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard1_replica1 as part of slice shard1 of collection solrj_collection on 127.0.0.1:56216_
[junit4] 2> 2134938 T5524 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard1_replica2 as part of slice shard1 of collection solrj_collection on 127.0.0.1:54524_
[junit4] 2> 2134940 T5524 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard2_replica1 as part of slice shard2 of collection solrj_collection on 127.0.0.1:42440_
[junit4] 2> 2134941 T5524 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard2_replica2 as part of slice shard2 of collection solrj_collection on 127.0.0.1:47176_
[junit4] 2> 2135008 T5466 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=solrj_collection_shard2_replica1&action=CREATE&numShards=2&collection=solrj_collection&shard=shard2&wt=javabin&version=2
[junit4] 2> 2135013 T5466 oasc.ZkController.publish publishing core=solrj_collection_shard2_replica1 state=down collection=solrj_collection
[junit4] 2> 2135014 T5466 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 2135014 T5452 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 2135016 T5453 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "core":"solrj_collection_shard2_replica1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:42440",
[junit4] 2> "node_name":"127.0.0.1:42440_",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"solrj_collection",
[junit4] 2> "operation":"state"} current state version: 1
[junit4] 2> 2135017 T5453 oasco.ReplicaMutator.updateState Update state numShards=2 message={
[junit4] 2> "core":"solrj_collection_shard2_replica1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:42440",
[junit4] 2> "node_name":"127.0.0.1:42440_",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"solrj_collection",
[junit4] 2> "operation":"state"}
[junit4] 2> 2135039 T5483 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=solrj_collection_shard2_replica2&action=CREATE&numShards=2&collection=solrj_collection&shard=shard2&wt=javabin&version=2
[junit4] 2> 2135041 T5483 oasc.ZkController.publish publishing core=solrj_collection_shard2_replica2 state=down collection=solrj_collection
[junit4] 2> 2135042 T5483 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 2135043 T5452 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 2135044 T5453 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "core":"solrj_collection_shard2_replica2",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:47176",
[junit4] 2> "node_name":"127.0.0.1:47176_",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"solrj_collection",
[junit4] 2> "operation":"state"} current state version: 1
[junit4] 2> 2135046 T5497 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=solrj_collection_shard1_replica2&action=CREATE&numShards=2&collection=solrj_collection&shard=shard1&wt=javabin&version=2
[junit4] 2> 2135046 T5453 oasco.ReplicaMutator.updateState Update state numShards=2 message={
[junit4] 2> "core":"solrj_collection_shard2_replica2",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:47176",
[junit4] 2> "node_name":"127.0.0.1:47176_",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"solrj_collection",
[junit4] 2> "operation":"state"}
[junit4] 2> 2135049 T5497 oasc.ZkController.publish publishing core=solrj_collection_shard1_replica2 state=down collection=solrj_collection
[junit4] 2> 2135050 T5452 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 2135050 T5497 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 2135051 T5453 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "core":"solrj_collection_shard1_replica2",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:54524",
[junit4] 2> "node_name":"127.0.0.1:54524_",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"solrj_collection",
[junit4] 2> "operation":"state"} current state version: 1
[junit4] 2> 2135052 T5453 oasco.ReplicaMutator.updateState Update state numShards=2 message={
[junit4] 2> "core":"solrj_collection_shard1_replica2",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:54524",
[junit4] 2> "node_name":"127.0.0.1:54524_",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"solrj_collection",
[junit4] 2> "operation":"state"}
[junit4] 2> 2135061 T5446 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=solrj_collection_shard1_replica1&action=CREATE&numShards=2&collection=solrj_collection&shard=shard1&wt=javabin&version=2
[junit4] 2> 2135062 T5446 oasc.ZkController.publish publishing core=solrj_collection_shard1_replica1 state=down collection=solrj_collection
[junit4] 2> 2135063 T5446 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 2135064 T5452 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 2135065 T5453 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "core":"solrj_collection_shard1_replica1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:56216",
[junit4] 2> "node_name":"127.0.0.1:56216_",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"solrj_collection",
[junit4] 2> "operation":"state"} current state version: 1
[junit4] 2> 2135066 T5453 oasco.ReplicaMutator.updateState Update state numShards=2 message={
[junit4] 2> "core":"solrj_collection_shard1_replica1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:56216",
[junit4] 2> "node_name":"127.0.0.1:56216_",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"solrj_collection",
[junit4] 2> "operation":"state"}
[junit4] 2> 2135169 T5506 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 2135169 T5522 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 2135169 T5452 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 2135169 T5459 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 2135169 T5490 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 2135169 T5474 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 2136015 T5466 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard2_replica1
[junit4] 2> 2136016 T5466 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
[junit4] 2> 2136016 T5466 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 2136017 T5466 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
[junit4] 2> 2136018 T5466 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 2136018 T5466 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-7269034E4F253D70-001/tempDir-003/solrj_collection_shard2_replica1/'
[junit4] 2> 2136037 T5466 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
[junit4] 2> 2136039 T5466 oasc.Config.<init> loaded config solrconfig.xml with version 0
[junit4] 2> 2136042 T5483 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard2_replica2
[junit4] 2> 2136043 T5483 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
[junit4] 2> 2136044 T5483 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 2136044 T5483 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
[junit4] 2> 2136045 T5483 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 2136046 T5483 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-7269034E4F253D70-001/tempDir-004/solrj_collection_shard2_replica2/'
[junit4] 2> 2136051 T5497 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard1_replica2
[junit4] 2> 2136051 T5497 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
[junit4] 2> 2136053 T5497 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 2136053 T5497 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
[junit4] 2> 2136055 T5497 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 2136056 T5497 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-7269034E4F253D70-001/tempDir-005/solrj_collection_shard1_replica2/'
[junit4] 2> 2136064 T5446 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard1_replica1
[junit4] 2> 2136064 T5446 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
[junit4] 2> 2136065 T5446 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 2136066 T5446 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
[junit4] 2> 2136068 T5446 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 2136069 T5446 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-7269034E4F253D70-001/tempDir-002/solrj_collection_shard1_replica1/'
[junit4] 2> 2136070 T5483 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
[junit4] 2> 2136075 T5483 oasc.Config.<init> loaded config solrconfig.xml with version 0
[junit4] 2> 2136080 T5497 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
[junit4] 2> 2136083 T5497 oasc.Config.<init> loaded config solrconfig.xml with version 0
[junit4] 2> 2136086 T5466 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
[junit4] 2> 2136093 T5446 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
[junit4] 2> 2136098 T5446 oasc.Config.<init> loaded config solrconfig.xml with version 0
[junit4] 2> 2136103 T5483 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
[junit4] 2> 2136140 T5466 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 2136141 T5497 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
[junit4] 2> 2136142 T5466 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
[junit4] 2> 2136148 T5483 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 2136149 T5446 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
[junit4] 2> 2136150 T5483 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
[junit4] 2> 2136169 T5497 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 2136173 T5497 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
[junit4] 2> 2136174 T5446 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 2136175 T5446 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
[junit4] 2> 2136206 T5466 oass.IndexSchema.readSchema [solrj_collection_shard2_replica1] Schema name=test
[junit4] 2> 2136212 T5483 oass.IndexSchema.readSchema [solrj_collection_shard2_replica2] Schema name=test
[junit4] 2> 2136232 T5446 oass.IndexSchema.readSchema [solrj_collection_shard1_replica1] Schema name=test
[junit4] 2> 2136247 T5497 oass.IndexSchema.readSchema [solrj_collection_shard1_replica2] Schema name=test
[junit4] 2> 2136460 T5466 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 2136471 T5466 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 2136474 T5466 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 2136477 T5497 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 2136488 T5497 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 2136491 T5497 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 2136495 T5483 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 2136500 T5466 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2136505 T5446 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 2136506 T5466 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2136506 T5483 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 2136509 T5483 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 2136511 T5466 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 2136512 T5466 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 2136524 T5466 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 2136525 T5466 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 2136526 T5466 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 2136526 T5466 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 2136527 T5466 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard2_replica1' using configuration from collection solrj_collection
[junit4] 2> 2136527 T5466 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 2136528 T5466 oasc.SolrCore.<init> [solrj_collection_shard2_replica1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-7269034E4F253D70-001/tempDir-003/solrj_collection_shard2_replica1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-7269034E4F253D70-001/tempDir-003/solrj_collection_shard2_replica1/data/
[junit4] 2> 2136529 T5466 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3c46f9
[junit4] 2> 2136529 T5446 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 2136532 T5446 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 2136533 T5497 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2136534 T5466 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest-7269034E4F253D70-001/tempDir-003/s
[...truncated too long message...]
] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 4) Thread[id=5678, name=zkCallback-751-thread-2, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 5) Thread[id=5609, name=TEST-CollectionsAPIDistributedZkTest.testDistribSearch-seed#[7269034E4F253D70]-EventThread, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)
[junit4] > 6) Thread[id=5610, name=zkCallback-751-thread-1, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > at __randomizedtesting.SeedInfo.seed([7269034E4F253D70]:0)Throwable #3: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=5706, name=zkCallback-751-thread-4, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 2) Thread[id=5608, name=TEST-CollectionsAPIDistributedZkTest.testDistribSearch-seed#[7269034E4F253D70]-SendThread(127.0.0.1:54370), state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] > 3) Thread[id=5685, name=zkCallback-751-thread-3, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 4) Thread[id=5678, name=zkCallback-751-thread-2, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 5) Thread[id=5610, name=zkCallback-751-thread-1, state=TIMED_WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > at __randomizedtesting.SeedInfo.seed([7269034E4F253D70]:0)
[junit4] Completed on J1 in 105.52s, 1 test, 2 failures, 2 errors <<< FAILURES!
[...truncated 673 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/build.xml:529: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/build.xml:477: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/build.xml:61: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/extra-targets.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build.xml:190: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/common-build.xml:508: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/common-build.xml:1363: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/common-build.xml:970: There were test failures: 452 suites, 1850 tests, 3 suite-level errors, 1 failure, 911 ignored (18 assumptions)
Total time: 76 minutes 19 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 32bit/jdk1.8.0_40-ea-b09 -client -XX:+UseG1GC (asserts: true)
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-5.x-Linux (64bit/jdk1.8.0_40-ea-b09) - Build
# 11603 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Linux/11603/
Java: 64bit/jdk1.8.0_40-ea-b09 -XX:+UseCompressedOops -XX:+UseParallelGC (asserts: true)
3 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([80AAE0A58F37295E]:0)
FAILED: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.testDistribSearch
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([80AAE0A58F37295E]:0)
FAILED: org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testDistribSearch
Error Message:
expected:<...AE0A58F37295E-001/te[mpDir-003/solrj_test_core_props_shard1_replica1]> but was:<...AE0A58F37295E-001/te[stPropertyParamsForCreate-001/instanceDir-zhld]>
Stack Trace:
org.junit.ComparisonFailure: expected:<...AE0A58F37295E-001/te[mpDir-003/solrj_test_core_props_shard1_replica1]> but was:<...AE0A58F37295E-001/te[stPropertyParamsForCreate-001/instanceDir-zhld]>
at __randomizedtesting.SeedInfo.seed([80AAE0A58F37295E:14C6EBDF8684962]:0)
at org.junit.Assert.assertEquals(Assert.java:125)
at org.junit.Assert.assertEquals(Assert.java:147)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.checkInstanceDirs(CollectionsAPIDistributedZkTest.java:1154)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:901)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:205)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869)
at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 9374 lines...]
[junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-80AAE0A58F37295E-001/init-core-data-001
[junit4] 2> 1495810 T6669 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (true)
[junit4] 2> 1495810 T6669 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /vakj/l
[junit4] 2> 1495813 T6669 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> 1495813 T6669 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1495814 T6670 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 1495914 T6669 oasc.ZkTestServer.run start zk server on port:34112
[junit4] 2> 1495914 T6669 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 1495915 T6669 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1495917 T6677 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@61af7e8c name:ZooKeeperConnection Watcher:127.0.0.1:34112 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1495918 T6669 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1495918 T6669 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 1495918 T6669 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 1495921 T6669 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 1495922 T6669 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1495923 T6680 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@339fd93e name:ZooKeeperConnection Watcher:127.0.0.1:34112/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1495923 T6669 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1495923 T6669 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 1495923 T6669 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 1495925 T6669 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 1495926 T6669 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 1495927 T6669 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 1495928 T6669 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 1495928 T6669 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 1495930 T6669 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 1495930 T6669 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 1495931 T6669 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1495931 T6669 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1495932 T6669 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 1495933 T6669 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 1495934 T6669 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 1495934 T6669 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 1495935 T6669 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 1495935 T6669 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 1495936 T6669 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 1495936 T6669 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 1495937 T6669 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 1495937 T6669 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 1495938 T6669 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1495939 T6669 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1495940 T6669 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 1495940 T6669 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 1495941 T6669 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 1495941 T6669 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> Using old style solr.xml
[junit4] 2> 1496161 T6669 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1496163 T6669 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:38236
[junit4] 2> 1496164 T6669 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 1496164 T6669 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 1496164 T6669 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-80AAE0A58F37295E-001/tempDir-002
[junit4] 2> 1496165 T6669 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-80AAE0A58F37295E-001/tempDir-002/'
[junit4] 2> 1496180 T6669 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-80AAE0A58F37295E-001/tempDir-002/solr.xml
[junit4] 2> 1496185 T6669 oasc.CoreContainer.<init> New CoreContainer 1098371661
[junit4] 2> 1496186 T6669 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-80AAE0A58F37295E-001/tempDir-002/]
[junit4] 2> 1496186 T6669 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 1496186 T6669 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
[junit4] 2> 1496187 T6669 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 1496187 T6669 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 1496187 T6669 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
[junit4] 2> 1496187 T6669 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 1496187 T6669 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 1496188 T6669 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 1496188 T6669 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 1496188 T6669 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 1496188 T6669 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=3400000&connTimeout=45000&retry=false
[junit4] 2> 1496189 T6669 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 1496189 T6669 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 1496189 T6669 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 1496189 T6669 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:34112/solr
[junit4] 2> 1496189 T6669 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 1496190 T6669 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 1496190 T6669 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1496191 T6692 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7a3bccb4 name:ZooKeeperConnection Watcher:127.0.0.1:34112 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1496192 T6669 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1496192 T6669 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 1496193 T6669 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1496194 T6695 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2d5980a7 name:ZooKeeperConnection Watcher:127.0.0.1:34112/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1496194 T6669 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1496195 T6669 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 1496196 T6669 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 1496198 T6669 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 1496199 T6669 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 1496200 T6669 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 1496201 T6669 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 1496202 T6669 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38236_vakj%2Fl
[junit4] 2> 1496202 T6669 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:38236_vakj%2Fl
[junit4] 2> 1496204 T6669 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 1496205 T6669 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 1496205 T6669 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 1496206 T6669 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:38236_vakj%2Fl
[junit4] 2> 1496206 T6669 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 1496207 T6669 oasc.Overseer.start Overseer (id=92993065774940163-127.0.0.1:38236_vakj%2Fl-n_0000000000) starting
[junit4] 2> 1496208 T6669 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 1496211 T6669 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 1496212 T6697 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 1496212 T6669 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 1496213 T6669 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 1496214 T6669 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1496215 T6696 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 1496217 T6669 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0
[junit4] 2> 1496217 T6669 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 1496217 T6669 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 1496218 T6669 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1496218 T6702 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3f65ca27 name:ZooKeeperConnection Watcher:127.0.0.1:34112/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1496219 T6669 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1496219 T6669 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 1496219 T6669 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1496221 T6669 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 1496221 T6669 oasc.AbstractFullDistribZkTestBase.createJettys Creating collection1 with stateFormat=2
[junit4] 2> 1496221 T6669 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 1496222 T6669 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1496222 T6705 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5205a2e0 name:ZooKeeperConnection Watcher:127.0.0.1:34112/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1496222 T6669 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1496223 T6669 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 1496224 T6695 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 1496225 T6696 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 0, message = {
[junit4] 2> "operation":"create",
[junit4] 2> "name":"collection1",
[junit4] 2> "numShards":"2",
[junit4] 2> "stateFormat":2} current state version: 0
[junit4] 2> 1496226 T6696 oasco.ClusterStateMutator.createCollection building a new cName: collection1
[junit4] 2> 1496226 T6696 oasco.ZkStateWriter.writePendingUpdates going to create_collection /collections/collection1/state.json
[junit4] 2> 1496227 T6695 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1496227 T6702 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1496464 T6669 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 1496465 T6669 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1496466 T6669 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49916
[junit4] 2> 1496467 T6669 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 1496467 T6669 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 1496468 T6669 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-80AAE0A58F37295E-001/tempDir-003
[junit4] 2> 1496468 T6669 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-80AAE0A58F37295E-001/tempDir-003/'
[junit4] 2> 1496478 T6669 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-80AAE0A58F37295E-001/tempDir-003/solr.xml
[junit4] 2> 1496484 T6669 oasc.CoreContainer.<init> New CoreContainer 1227869626
[junit4] 2> 1496484 T6669 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-80AAE0A58F37295E-001/tempDir-003/]
[junit4] 2> 1496484 T6669 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 1496485 T6669 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
[junit4] 2> 1496485 T6669 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 1496485 T6669 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 1496485 T6669 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
[junit4] 2> 1496485 T6669 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 1496486 T6669 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 1496486 T6669 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 1496486 T6669 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 1496486 T6669 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 1496486 T6669 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=3400000&connTimeout=45000&retry=false
[junit4] 2> 1496487 T6669 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 1496487 T6669 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 1496487 T6669 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 1496487 T6669 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:34112/solr
[junit4] 2> 1496488 T6669 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 1496488 T6669 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 1496488 T6669 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1496489 T6717 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1bcff1cd name:ZooKeeperConnection Watcher:127.0.0.1:34112 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1496490 T6669 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1496490 T6669 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 1496492 T6669 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1496492 T6720 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7646d69e name:ZooKeeperConnection Watcher:127.0.0.1:34112/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1496492 T6669 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1496495 T6669 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1497498 T6669 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49916_vakj%2Fl
[junit4] 2> 1497498 T6669 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49916_vakj%2Fl
[junit4] 2> 1497501 T6669 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 1497503 T6669 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0
[junit4] 2> 1497504 T6669 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 1497733 T6669 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 1497734 T6669 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1497735 T6669 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:55331
[junit4] 2> 1497736 T6669 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 1497736 T6669 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 1497737 T6669 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-80AAE0A58F37295E-001/tempDir-004
[junit4] 2> 1497737 T6669 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-80AAE0A58F37295E-001/tempDir-004/'
[junit4] 2> 1497747 T6669 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-80AAE0A58F37295E-001/tempDir-004/solr.xml
[junit4] 2> 1497753 T6669 oasc.CoreContainer.<init> New CoreContainer 971337882
[junit4] 2> 1497753 T6669 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-80AAE0A58F37295E-001/tempDir-004/]
[junit4] 2> 1497753 T6669 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 1497754 T6669 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
[junit4] 2> 1497754 T6669 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 1497754 T6669 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 1497754 T6669 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
[junit4] 2> 1497754 T6669 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 1497755 T6669 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 1497755 T6669 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 1497755 T6669 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 1497755 T6669 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 1497756 T6669 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=3400000&connTimeout=45000&retry=false
[junit4] 2> 1497756 T6669 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 1497756 T6669 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 1497756 T6669 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 1497756 T6669 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:34112/solr
[junit4] 2> 1497757 T6669 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 1497757 T6669 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 1497758 T6669 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1497759 T6733 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@493cf240 name:ZooKeeperConnection Watcher:127.0.0.1:34112 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1497760 T6669 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1497760 T6669 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 1497761 T6669 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1497762 T6736 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@15cd36ed name:ZooKeeperConnection Watcher:127.0.0.1:34112/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1497762 T6669 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1497765 T6669 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1498768 T6669 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55331_vakj%2Fl
[junit4] 2> 1498768 T6669 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:55331_vakj%2Fl
[junit4] 2> 1498771 T6669 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 1498772 T6669 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0
[junit4] 2> 1498773 T6669 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 1498991 T6669 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
[junit4] 2> 1498992 T6669 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1498993 T6669 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:57288
[junit4] 2> 1498994 T6669 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 1498994 T6669 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 1498994 T6669 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-80AAE0A58F37295E-001/tempDir-005
[junit4] 2> 1498995 T6669 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-80AAE0A58F37295E-001/tempDir-005/'
[junit4] 2> 1499004 T6669 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-80AAE0A58F37295E-001/tempDir-005/solr.xml
[junit4] 2> 1499011 T6669 oasc.CoreContainer.<init> New CoreContainer 1549896045
[junit4] 2> 1499011 T6669 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-80AAE0A58F37295E-001/tempDir-005/]
[junit4] 2> 1499011 T6669 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 1499012 T6669 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
[junit4] 2> 1499012 T6669 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 1499012 T6669 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 1499012 T6669 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
[junit4] 2> 1499012 T6669 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 1499013 T6669 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 1499013 T6669 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 1499013 T6669 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 1499013 T6669 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 1499014 T6669 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=3400000&connTimeout=45000&retry=false
[junit4] 2> 1499014 T6669 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 1499014 T6669 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 1499014 T6669 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 1499014 T6669 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:34112/solr
[junit4] 2> 1499015 T6669 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 1499015 T6669 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 1499015 T6669 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1499016 T6749 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@505b9397 name:ZooKeeperConnection Watcher:127.0.0.1:34112 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1499017 T6669 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1499017 T6669 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 1499019 T6669 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1499019 T6752 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@52ce4ba8 name:ZooKeeperConnection Watcher:127.0.0.1:34112/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1499020 T6669 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1499022 T6669 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1500025 T6669 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57288_vakj%2Fl
[junit4] 2> 1500025 T6669 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57288_vakj%2Fl
[junit4] 2> 1500028 T6669 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 1500030 T6669 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0
[junit4] 2> 1500031 T6669 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 1500254 T6669 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 4
[junit4] 2> 1500254 T6669 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 1500256 T6669 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:45339
[junit4] 2> 1500257 T6669 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 1500257 T6669 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 1500257 T6669 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-80AAE0A58F37295E-001/tempDir-006
[junit4] 2> 1500257 T6669 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-80AAE0A58F37295E-001/tempDir-006/'
[junit4] 2> 1500267 T6669 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-80AAE0A58F37295E-001/tempDir-006/solr.xml
[junit4] 2> 1500274 T6669 oasc.CoreContainer.<init> New CoreContainer 773581133
[junit4] 2> 1500274 T6669 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-80AAE0A58F37295E-001/tempDir-006/]
[junit4] 2> 1500275 T6669 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4] 2> 1500275 T6669 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
[junit4] 2> 1500275 T6669 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 1500275 T6669 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 1500276 T6669 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
[junit4] 2> 1500276 T6669 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 1500276 T6669 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 1500276 T6669 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 1500276 T6669 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 1500277 T6669 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 1500277 T6669 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=3400000&connTimeout=45000&retry=false
[junit4] 2> 1500277 T6669 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 1500277 T6669 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 1500278 T6669 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 1500278 T6669 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:34112/solr
[junit4] 2> 1500278 T6669 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 1500278 T6669 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
[junit4] 2> 1500279 T6669 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1500280 T6765 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@674c710f name:ZooKeeperConnection Watcher:127.0.0.1:34112 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1500281 T6669 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1500282 T6669 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
[junit4] 2> 1500284 T6669 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1500285 T6768 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@28b4f92e name:ZooKeeperConnection Watcher:127.0.0.1:34112/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1500285 T6669 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1500288 T6669 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1501290 T6669 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45339_vakj%2Fl
[junit4] 2> 1501291 T6669 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:45339_vakj%2Fl
[junit4] 2> 1501294 T6669 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 1501297 T6669 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0
[junit4] 2> 1501298 T6669 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 1501302 T6713 oasha.CollectionsHandler.handleCreateAction Creating Collection : replicationFactor=2&collection.configName=conf1&router.field=myOwnField&name=solrj_collection&action=CREATE&numShards=2&wt=javabin&version=2
[junit4] 2> 1501303 T6695 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/collection-queue-work state SyncConnected
[junit4] 2> 1501304 T6697 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
[junit4] 2> "operation":"create",
[junit4] 2> "fromApi":"true",
[junit4] 2> "name":"solrj_collection",
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "collection.configName":"conf1",
[junit4] 2> "numShards":"2",
[junit4] 2> "router.field":"myOwnField"}
[junit4] 2> 1501305 T6770 oasc.OverseerCollectionProcessor.processMessage WARN OverseerCollectionProcessor.processMessage : create , {
[junit4] 2> "operation":"create",
[junit4] 2> "fromApi":"true",
[junit4] 2> "name":"solrj_collection",
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "collection.configName":"conf1",
[junit4] 2> "numShards":"2",
[junit4] 2> "router.field":"myOwnField"}
[junit4] 2> 1501306 T6770 oasc.OverseerCollectionProcessor.createConfNode creating collections conf node /collections/solrj_collection
[junit4] 2> 1501307 T6770 oascc.SolrZkClient.makePath makePath: /collections/solrj_collection
[junit4] 2> 1501309 T6695 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 1501310 T6696 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"create",
[junit4] 2> "fromApi":"true",
[junit4] 2> "name":"solrj_collection",
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "collection.configName":"conf1",
[junit4] 2> "numShards":"2",
[junit4] 2> "router.field":"myOwnField"} current state version: 1
[junit4] 2> 1501310 T6696 oasco.ClusterStateMutator.createCollection building a new cName: solrj_collection
[junit4] 2> 1501311 T6768 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 1501311 T6752 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 1501311 T6736 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 1501311 T6702 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 1501311 T6720 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 1501311 T6695 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 1501509 T6770 oasc.OverseerCollectionProcessor.createCollection Creating SolrCores for new collection solrj_collection, shardNames [shard1, shard2] , replicationFactor : 2
[junit4] 2> 1501510 T6770 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard1_replica1 as part of slice shard1 of collection solrj_collection on 127.0.0.1:57288_vakj%2Fl
[junit4] 2> 1501510 T6770 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard1_replica2 as part of slice shard1 of collection solrj_collection on 127.0.0.1:38236_vakj%2Fl
[junit4] 2> 1501511 T6770 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard2_replica1 as part of slice shard2 of collection solrj_collection on 127.0.0.1:55331_vakj%2Fl
[junit4] 2> 1501512 T6686 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=solrj_collection_shard1_replica2&action=CREATE&numShards=2&collection=solrj_collection&shard=shard1&wt=javabin&version=2
[junit4] 2> 1501512 T6770 oasc.OverseerCollectionProcessor.createCollection Creating shard solrj_collection_shard2_replica2 as part of slice shard2 of collection solrj_collection on 127.0.0.1:49916_vakj%2Fl
[junit4] 2> 1501512 T6746 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=solrj_collection_shard1_replica1&action=CREATE&numShards=2&collection=solrj_collection&shard=shard1&wt=javabin&version=2
[junit4] 2> 1501513 T6728 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=solrj_collection_shard2_replica1&action=CREATE&numShards=2&collection=solrj_collection&shard=shard2&wt=javabin&version=2
[junit4] 2> 1501514 T6686 oasc.ZkController.publish publishing core=solrj_collection_shard1_replica2 state=down collection=solrj_collection
[junit4] 2> 1501514 T6728 oasc.ZkController.publish publishing core=solrj_collection_shard2_replica1 state=down collection=solrj_collection
[junit4] 2> 1501514 T6695 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 1501514 T6686 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 1501515 T6711 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&collection.configName=conf1&name=solrj_collection_shard2_replica2&action=CREATE&numShards=2&collection=solrj_collection&shard=shard2&wt=javabin&version=2
[junit4] 2> 1501515 T6728 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 1501514 T6746 oasc.ZkController.publish publishing core=solrj_collection_shard1_replica1 state=down collection=solrj_collection
[junit4] 2> 1501515 T6696 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "core":"solrj_collection_shard1_replica2",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:38236/vakj/l",
[junit4] 2> "node_name":"127.0.0.1:38236_vakj%2Fl",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"solrj_collection",
[junit4] 2> "operation":"state"} current state version: 2
[junit4] 2> 1501516 T6746 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 1501516 T6696 oasco.ReplicaMutator.updateState Update state numShards=2 message={
[junit4] 2> "core":"solrj_collection_shard1_replica2",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:38236/vakj/l",
[junit4] 2> "node_name":"127.0.0.1:38236_vakj%2Fl",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"solrj_collection",
[junit4] 2> "operation":"state"}
[junit4] 2> 1501517 T6696 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 2, message = {
[junit4] 2> "core":"solrj_collection_shard2_replica1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:55331/vakj/l",
[junit4] 2> "node_name":"127.0.0.1:55331_vakj%2Fl",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"solrj_collection",
[junit4] 2> "operation":"state"} current state version: 2
[junit4] 2> 1501518 T6696 oasco.ReplicaMutator.updateState Update state numShards=2 message={
[junit4] 2> "core":"solrj_collection_shard2_replica1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:55331/vakj/l",
[junit4] 2> "node_name":"127.0.0.1:55331_vakj%2Fl",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"solrj_collection",
[junit4] 2> "operation":"state"}
[junit4] 2> 1501519 T6695 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 1501519 T6711 oasc.ZkController.publish publishing core=solrj_collection_shard2_replica2 state=down collection=solrj_collection
[junit4] 2> 1501519 T6696 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 2, message = {
[junit4] 2> "core":"solrj_collection_shard1_replica1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:57288/vakj/l",
[junit4] 2> "node_name":"127.0.0.1:57288_vakj%2Fl",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"solrj_collection",
[junit4] 2> "operation":"state"} current state version: 2
[junit4] 2> 1501520 T6711 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 1501520 T6696 oasco.ReplicaMutator.updateState Update state numShards=2 message={
[junit4] 2> "core":"solrj_collection_shard1_replica1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:57288/vakj/l",
[junit4] 2> "node_name":"127.0.0.1:57288_vakj%2Fl",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"solrj_collection",
[junit4] 2> "operation":"state"}
[junit4] 2> 1501521 T6696 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "core":"solrj_collection_shard2_replica2",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:49916/vakj/l",
[junit4] 2> "node_name":"127.0.0.1:49916_vakj%2Fl",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"solrj_collection",
[junit4] 2> "operation":"state"} current state version: 2
[junit4] 2> 1501522 T6696 oasco.ReplicaMutator.updateState Update state numShards=2 message={
[junit4] 2> "core":"solrj_collection_shard2_replica2",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:49916/vakj/l",
[junit4] 2> "node_name":"127.0.0.1:49916_vakj%2Fl",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"solrj_collection",
[junit4] 2> "operation":"state"}
[junit4] 2> 1501523 T6695 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 1501523 T6768 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 1501523 T6720 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 1501524 T6736 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 1501524 T6702 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 1501523 T6695 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 1501523 T6752 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4] 2> 1502515 T6686 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard1_replica2
[junit4] 2> 1502515 T6686 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
[junit4] 2> 1502515 T6728 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard2_replica1
[junit4] 2> 1502516 T6728 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
[junit4] 2> 1502516 T6686 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 1502516 T6686 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
[junit4] 2> 1502516 T6746 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard1_replica1
[junit4] 2> 1502516 T6728 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 1502517 T6728 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
[junit4] 2> 1502517 T6746 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
[junit4] 2> 1502517 T6686 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1502518 T6686 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-80AAE0A58F37295E-001/tempDir-002/solrj_collection_shard1_replica2/'
[junit4] 2> 1502520 T6711 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for solrj_collection_shard2_replica2
[junit4] 2> 1502521 T6711 oasc.ZkController.createCollectionZkNode Check for collection zkNode:solrj_collection
[junit4] 2> 1502521 T6746 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 1502521 T6746 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
[junit4] 2> 1502522 T6711 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 1502522 T6711 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
[junit4] 2> 1502521 T6728 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1502523 T6728 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-80AAE0A58F37295E-001/tempDir-004/solrj_collection_shard2_replica1/'
[junit4] 2> 1502523 T6711 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1502523 T6711 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-80AAE0A58F37295E-001/tempDir-003/solrj_collection_shard2_replica2/'
[junit4] 2> 1502523 T6746 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1502524 T6746 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-80AAE0A58F37295E-001/tempDir-005/solrj_collection_shard1_replica1/'
[junit4] 2> 1502532 T6686 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
[junit4] 2> 1502533 T6686 oasc.Config.<init> loaded config solrconfig.xml with version 0
[junit4] 2> 1502538 T6728 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
[junit4] 2> 1502542 T6711 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
[junit4] 2> 1502543 T6728 oasc.Config.<init> loaded config solrconfig.xml with version 0
[junit4] 2> 1502544 T6686 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
[junit4] 2> 1502546 T6746 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
[junit4] 2> 1502547 T6711 oasc.Config.<init> loaded config solrconfig.xml with version 0
[junit4] 2> 1502552 T6728 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
[junit4] 2> 1502555 T6686 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1502556 T6686 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
[junit4] 2> 1502563 T6728 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1502563 T6686 oass.IndexSchema.readSchema [solrj_collection_shard1_replica2] Schema name=test
[junit4] 2> 1502564 T6728 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
[junit4] 2> 1502571 T6711 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
[junit4] 2> 1502574 T6746 oasc.Config.<init> loaded config solrconfig.xml with version 0
[junit4] 2> 1502582 T6728 oass.IndexSchema.readSchema [solrj_collection_shard2_replica1] Schema name=test
[junit4] 2> 1502603 T6746 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
[junit4] 2> 1502616 T6746 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1502618 T6746 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
[junit4] 2> 1502619 T6711 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 1502620 T6711 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
[junit4] 2> 1502626 T6746 oass.IndexSchema.readSchema [solrj_collection_shard1_replica1] Schema name=test
[junit4] 2> 1502627 T6711 oass.IndexSchema.readSchema [solrj_collection_shard2_replica2] Schema name=test
[junit4] 2> 1502724 T6686 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 1502729 T6686 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 1502731 T6686 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 1502738 T6686 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1502740 T6686 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1502743 T6686 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1502744 T6686 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1502749 T6686 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1502749 T6686 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1502750 T6686 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1502750 T6686 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1502751 T6686 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard1_replica2' using configuration from collection solrj_collection
[junit4] 2> 1502751 T6686 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 1502752 T6686 oasc.SolrCore.<init> [solrj_collection_shard1_replica2] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-80AAE0A58F37295E-001/tempDir-002/solrj_collection_shard1_replica2/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-80AAE0A58F37295E-001/tempDir-002/solrj_collection_shard1_replica2/data/
[junit4] 2> 1502753 T6686 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@e1a6bd
[junit4] 2> 1502754 T6686 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-80AAE0A58F37295E-001/tempDir-002/solrj_collection_shard1_replica2/data
[junit4] 2> 1502755 T6686 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-80AAE0A58F37295E-001/tempDir-002/solrj_collection_shard1_replica2/data/index/
[junit4] 2> 1502755 T6686 oasc.SolrCore.initIndex WARN [solrj_collection_shard1_replica2] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-80AAE0A58F37295E-001/tempDir-002/solrj_collection_shard1_replica2/data/index' doesn't exist. Creating new index...
[junit4] 2> 1502756 T6686 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-80AAE0A58F37295E-001/tempDir-002/solrj_collection_shard1_replica2/data/index
[junit4] 2> 1502757 T6686 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=15, maxMergeAtOnceExplicit=27, maxMergedSegmentMB=58.8857421875, floorSegmentMB=0.275390625, forceMergeDeletesPctAllowed=13.360653399670893, segmentsPerTier=40.0, maxCFSSegmentSizeMB=1.740234375, noCFSRatio=0.5013962192368391
[junit4] 2> 1502758 T6686 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@2ca8f3c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@766a5dd4),segFN=segments_1,generation=1}
[junit4] 2> 1502759 T6686 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1502764 T6686 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 1502765 T6686 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 1502766 T6686 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 1502766 T6686 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 1502767 T6686 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 1502767 T6686 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 1502768 T6686 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1502768 T6711 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 1502769 T6686 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1502779 T6686 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 1502779 T6686 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1502780 T6686 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1502780 T6686 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1502781 T6686 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1502781 T6686 oasc.RequestHandlers.initHandlersFromConfig created /config: org.apache.solr.handler.SolrConfigHandler
[junit4] 2> 1502782 T6686 oasc.RequestHandlers.initHandlersFromConfig created /schema: org.apache.solr.handler.SchemaHandler
[junit4] 2> 1502782 T6711 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 1502782 T6686 oasc.RequestHandlers.initHandlersFromConfig created /replication: org.apache.solr.handler.ReplicationHandler
[junit4] 2> 1502783 T6686 oasc.RequestHandlers.initHandlersFromConfig created /get: org.apache.solr.handler.RealTimeGetHandler
[junit4] 2> 1502783 T6686 oasc.RequestHandlers.initHandlersFromConfig created /admin/luke: org.apache.solr.handler.admin.LukeRequestHandler
[junit4] 2> 1502784 T6711 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 1502784 T6686 oasc.RequestHandlers.initHandlersFromConfig created /admin/system: org.apache.solr.handler.admin.SystemInfoHandler
[junit4] 2> 1502785 T6686 oasc.RequestHandlers.initHandlersFromConfig created /admin/mbeans: org.apache.solr.handler.admin.SolrInfoMBeanHandler
[junit4] 2> 1502785 T6686 oasc.RequestHandlers.initHandlersFromConfig created /admin/plugins: org.apache.solr.handler.admin.PluginInfoHandler
[junit4] 2> 1502786 T6686 oasc.RequestHandlers.initHandlersFromConfig created /admin/threads: org.apache.solr.handler.admin.ThreadDumpHandler
[junit4] 2> 1502786 T6686 oasc.RequestHandlers.initHandlersFromConfig created /admin/properties: org.apache.solr.handler.admin.PropertiesRequestHandler
[junit4] 2> 1502787 T6686 oasc.RequestHandlers.initHandlersFromConfig created /admin/logging: org.apache.solr.handler.admin.LoggingHandler
[junit4] 2> 1502787 T6686 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: org.apache.solr.handler.PingRequestHandler
[junit4] 2> 1502788 T6686 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 1502788 T6686 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 1502788 T6686 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 1502791 T6686 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1502792 T6686 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1502793 T6728 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 1502793 T6686 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1502794 T6686 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 1502796 T6686 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 1502796 T6686 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 1502797 T6728 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 1502799 T6686 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1879302735, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 1502799 T6711 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1502799 T6686 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@2ca8f3c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@766a5dd4),segFN=segments_1,generation=1}
[junit4] 2> 1502800 T6686 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1502800 T6728 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 1502800 T6686 oass.SolrIndexSearcher.<init> Opening Searcher@7333eb59[solrj_collection_shard1_replica2] main
[junit4] 2> 1502801 T6686 oascc.ZkStateReader.readConfigName Load collection config from:/collections/solrj_collection
[junit4] 2> 1502801 T6686 oascc.ZkStateReader.readConfigName path=/collections/solrj_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 1502802 T6686 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 1502802 T6711 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1502802 T6686 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1502803 T6686 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 1502803 T6686 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1502803 T6686 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 1502804 T6686 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1502804 T6686 oascc.SolrZkClient.makePath makePath: /configs/conf1/_rest_managed.json
[junit4] 2> 1502806 T6711 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1502807 T6711 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1502807 T6686 oasr.ManagedResourceStorage$ZooKeeperStorageIO$1.close Wrote 38 bytes to new znode /configs/conf1/_rest_managed.json
[junit4] 2> 1502809 T6711 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1502810 T6686 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1502810 T6746 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 1502810 T6711 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1502810 T6686 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 1502812 T6711 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1502812 T6711 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1502812 T6686 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4] 2> 1502812 T6711 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard2_replica2' using configuration from collection solrj_collection
[junit4] 2> 1502813 T6686 oasc.CoreContainer.registerCore registering core: solrj_collection_shard1_replica2
[junit4] 2> 1502813 T6686 oasc.ZkController.register Register replica - core:solrj_collection_shard1_replica2 address:http://127.0.0.1:38236/vakj/l collection:solrj_collection shard:shard1
[junit4] 2> 1502812 T6728 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1502813 T6711 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 1502814 T6711 oasc.SolrCore.<init> [solrj_collection_shard2_replica2] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-80AAE0A58F37295E-001/tempDir-003/solrj_collection_shard2_replica2/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-80AAE0A58F37295E-001/tempDir-003/solrj_collection_shard2_replica2/data/
[junit4] 2> 1502815 T6711 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@e1a6bd
[junit4] 2> 1502814 T6686 oascc.SolrZkClient.makePath makePath: /collections/solrj_collection/leader_elect/shard1/election
[junit4] 2> 1502815 T6746 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 1502815 T6711 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-80AAE0A58F37295E-001/tempDir-003/solrj_collection_shard2_replica2/data
[junit4] 2> 1502816 T6728 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1502816 T6711 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-80AAE0A58F37295E-001/tempDir-003/solrj_collection_shard2_replica2/data/index/
[junit4] 2> 1502816 T6711 oasc.SolrCore.initIndex WARN [solrj_collection_shard2_replica2] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-80AAE0A58F37295E-001/tempDir-003/solrj_collection_shard2_replica2/data/index' doesn't exist. Creating new index...
[junit4] 2> 1502817 T6711 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-80AAE0A58F37295E-001/tempDir-003/solrj_collection_shard2_replica2/data/index
[junit4] 2> 1502817 T6746 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 1502817 T6711 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=15, maxMergeAtOnceExplicit=27, maxMergedSegmentMB=58.8857421875, floorSegmentMB=0.275390625, forceMergeDeletesPctAllowed=13.360653399670893, segmentsPerTier=40.0, maxCFSSegmentSizeMB=1.740234375, noCFSRatio=0.5013962192368391
[junit4] 2> 1502818 T6775 oasc.SolrCore.registerSearcher [solrj_collection_shard1_replica2] Registered new searcher Searcher@7333eb59[solrj_collection_shard1_replica2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1502818 T6728 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1502818 T6711 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@2f9a94e8 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@34b975fa),segFN=segments_1,generation=1}
[junit4] 2> 1502819 T6711 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1502819 T6728 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1502819 T6728 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1502820 T6728 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1502820 T6728 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1502821 T6728 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1502821 T6686 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 1502821 T6728 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard2_replica1' using configuration from collection solrj_collection
[junit4] 2> 1502821 T6728 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 1502822 T6728 oasc.SolrCore.<init> [solrj_collection_shard2_replica1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-80AAE0A58F37295E-001/tempDir-004/solrj_collection_shard2_replica1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-80AAE0A58F37295E-001/tempDir-004/solrj_collection_shard2_replica1/data/
[junit4] 2> 1502822 T6711 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 1502822 T6695 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 1502822 T6728 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@e1a6bd
[junit4] 2> 1502823 T6696 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"solrj_collection"} current state version: 3
[junit4] 2> 1502822 T6711 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 1502824 T6711 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 1502824 T6711 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 1502824 T6711 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 1502825 T6711 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 1502825 T6711 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1502825 T6711 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 1502826 T6711 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 1502826 T6711 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1502827 T6711 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1502827 T6711 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1502827 T6746 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1502822 T6686 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=0ms
[junit4] 2> 1502828 T6686 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Was waiting for replicas to come up, but they are taking too long - assuming they won't come back till later
[junit4] 2> 1502828 T6686 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C2745 name=solrj_collection_shard1_replica2 org.apache.solr.core.SolrCore@795ad2d0 url=http://127.0.0.1:38236/vakj/l/solrj_collection_shard1_replica2 node=127.0.0.1:38236_vakj%2Fl C2745_STATE=coll:solrj_collection core:solrj_collection_shard1_replica2 props:{core=solrj_collection_shard1_replica2, base_url=http://127.0.0.1:38236/vakj/l, node_name=127.0.0.1:38236_vakj%2Fl, state=down}
[junit4] 2> 1502828 T6686 C2745 P38236 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:38236/vakj/l/solrj_collection_shard1_replica2/
[junit4] 2> 1502829 T6686 C2745 P38236 oasu.PeerSync.sync PeerSync: core=solrj_collection_shard1_replica2 url=http://127.0.0.1:38236/vakj/l START replicas=[http://127.0.0.1:57288/vakj/l/solrj_collection_shard1_replica1/] nUpdates=100
[junit4] 2> 1502827 T6711 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 1502824 T6728 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-80AAE0A58F37295E-001/tempDir-004/solrj_collection_shard2_replica1/data
[junit4] 2> 1502831 T6746 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 1502829 T6686 C2745 P38236 oasu.PeerSync.sync PeerSync: core=solrj_collection_shard1_replica2 url=http://127.0.0.1:38236/vakj/l DONE. We have no versions. sync failed.
[junit4] 2> 1502831 T6728 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-80AAE0A58F37295E-001/tempDir-004/solrj_collection_shard2_replica1/data/index/
[junit4] 2> 1502832 T6711 oasc.RequestHandlers.initHandlersFromConfig created /config: org.apache.solr.handler.SolrConfigHandler
[junit4] 2> 1502832 T6686 C2745 P38236 oasc.SyncStrategy.syncReplicas Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 1502832 T6728 oasc.SolrCore.initIndex WARN [solrj_collection_shard2_replica1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-80AAE0A58F37295E-001/tempDir-004/solrj_collection_shard2_replica1/data/index' doesn't exist. Creating new index...
[junit4] 2> 1502833 T6711 oasc.RequestHandlers.initHandlersFromConfig created /schema: org.apache.solr.handler.SchemaHandler
[junit4] 2> 1502833 T6728 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPIDistributedZkTest-80AAE0A58F37295E-001/tempDir-004/solrj_collection_shard2_replica1/data/index
[junit4] 2> 1502833 T6746 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1502833 T6686 oasc.ShardLeaderElectionContext.runLeaderProcess We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 1502834 T6728 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=15, maxMergeAtOnceExplicit=27, maxMergedSegmentMB=58.8857421875, floorSegmentMB=0.275390625, forceMergeDeletesPctAllowed=13.360653399670893, segmentsPerTier=40.0, maxCFSSegmentSizeMB=1.740234375, noCFSRatio=0.5013962192368391
[junit4] 2> 1502834 T6746 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1502835 T6746 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1502835 T6746 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 1502834 T6711 oasc.RequestHandlers.initHandlersFromConfig created /replication: org.apache.solr.handler.ReplicationHandler
[junit4] 2> 1502835 T6728 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=MockDirectoryWrapper(RAMDirectory@2c1386d5 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4ef797fc),segFN=segments_1,generation=1}
[junit4] 2> 1502836 T6711 oasc.RequestHandlers.initHandlersFromConfig created /get: org.apache.solr.handler.RealTimeGetHandler
[junit4] 2> 1502834 T6686 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:38236/vakj/l/solrj_collection_shard1_replica2/ shard1
[junit4] 2> 1502836 T6711 oasc.RequestHandlers.initHandlersFromConfig created /admin/luke: org.apache.solr.handler.admin.LukeRequestHandler
[junit4] 2> 1502837 T6686 oascc.SolrZkClient.makePath makePath: /collections/solrj_collection/leaders/shard1
[junit4] 2> 1502836 T6728 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 1502836 T6746 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 1502838 T6746 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 1502837 T6711 oasc.RequestHandlers.initHandlersFromConfig created /admin/system: org.apache.solr.handler.admin.SystemInfoHandler
[junit4] 2> 1502839 T6746 oasc.CoreContainer.create Creating SolrCore 'solrj_collection_shard1_replica1' using configuration from collection solrj_collection
[junit4] 2> 1502839 T6711 oasc.RequestHandlers.initHandlersFromConfig created /admin/mbeans: org.apache.solr.handler.admin.SolrInfoMBeanHandler
[junit4] 2> 1502840 T6695 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
[junit4] 2> 1502839 T6746 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 1502840 T6711 oasc.RequestHandlers.initHandlersFromConfig created /admin/plugins: org.apache.solr.handler.admin.PluginInfoHandler
[junit4] 2> 1502840 T6696 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"solrj_collection",
[junit4] 2> "base_url":"http://127.0.0.1:38236/vakj/l",
[junit4] 2> "core":"solrj_collection_shard1_replica2",
[junit4] 2> "state":"active"} current state version: 3
[junit4] 2> 1502840 T6728 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorC
[...truncated too long message...]
va:745)
[junit4] 2> 25) Thread[id=1128, name=zkCallback-65-thread-115, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderTest]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] 2> at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] 2> at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] 2> at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 26) Thread[id=442, name=TEST-ChaosMonkeySafeLeaderTest.testDistribSearch-seed#[80AAE0A58F37295E]-SendThread(127.0.0.1:54208), state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderTest]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:994)
[junit4] 2> 27) Thread[id=474, name=commitScheduler-237-thread-1, state=WAITING, group=TGRP-ChaosMonkeySafeLeaderTest]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)
[junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 28) Thread[id=413, name=qtp1548544683-413 Acceptor0 SslSelectChannelConnector@127.0.0.1:47405, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderTest]
[junit4] 2> at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:956)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 29) Thread[id=447, name=Thread-203, state=WAITING, group=TGRP-ChaosMonkeySafeLeaderTest]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Object.wait(Object.java:502)
[junit4] 2> at org.apache.solr.core.CloserThread.run(CoreContainer.java:924)
[junit4] 2> 30) Thread[id=453, name=qtp1702397052-453 Acceptor0 SslSelectChannelConnector@127.0.0.1:55621, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderTest]
[junit4] 2> at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:956)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 31) Thread[id=467, name=Thread-214, state=WAITING, group=TGRP-ChaosMonkeySafeLeaderTest]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Object.wait(Object.java:502)
[junit4] 2> at org.apache.solr.core.CloserThread.run(CoreContainer.java:924)
[junit4] 2> 32) Thread[id=422, name=TEST-ChaosMonkeySafeLeaderTest.testDistribSearch-seed#[80AAE0A58F37295E]-SendThread(127.0.0.1:54208), state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderTest]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:101)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:940)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
[junit4] 2> 33) Thread[id=451, name=qtp1702397052-451 Selector0, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderTest]
[junit4] 2> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
[junit4] 2> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
[junit4] 2> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
[junit4] 2> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
[junit4] 2> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
[junit4] 2> at org.eclipse.jetty.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:569)
[junit4] 2> at org.eclipse.jetty.io.nio.SelectorManager$1.run(SelectorManager.java:290)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 34) Thread[id=476, name=commitScheduler-255-thread-1, state=WAITING, group=TGRP-ChaosMonkeySafeLeaderTest]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)
[junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene50): {rnd_b=PostingsFormat(name=Asserting), _version_=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), a_t=Lucene50(blocksize=128), a_i=PostingsFormat(name=Asserting), id=PostingsFormat(name=Asserting)}, docValues:{}, sim=DefaultSimilarity, locale=ar_KW, timezone=Etc/GMT-12
[junit4] 2> NOTE: Linux 3.13.0-43-generic amd64/Oracle Corporation 1.8.0_40-ea (64-bit)/cpus=8,threads=40,free=88675384,total=267386880
[junit4] 2> NOTE: All tests run in this JVM: [TestCloudManagedSchema, CoreAdminRequestStatusTest, TestSolrJ, StatelessScriptUpdateProcessorFactoryTest, TestNonNRTOpen, TestManagedSchemaFieldTypeResource, TestDistributedMissingSort, SimpleMLTQParserTest, ScriptEngineTest, SampleTest, URLClassifyProcessorTest, DocExpirationUpdateProcessorFactoryTest, TermsComponentTest, ChaosMonkeySafeLeaderTest]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=ChaosMonkeySafeLeaderTest -Dtests.seed=80AAE0A58F37295E -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ar_KW -Dtests.timezone=Etc/GMT-12 -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] ERROR 0.00s J1 | ChaosMonkeySafeLeaderTest (suite) <<<
[junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4] > at __randomizedtesting.SeedInfo.seed([80AAE0A58F37295E]:0)
[junit4] Completed on J1 in 7223.64s, 1 test, 2 errors <<< FAILURES!
[...truncated 309 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/build.xml:529: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/build.xml:477: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/build.xml:61: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/extra-targets.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build.xml:190: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/common-build.xml:508: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/common-build.xml:1363: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/common-build.xml:970: There were test failures: 452 suites, 1850 tests, 1 suite-level error, 1 error, 1 failure, 487 ignored (16 assumptions)
Total time: 154 minutes 4 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 64bit/jdk1.8.0_40-ea-b09 -XX:+UseCompressedOops -XX:+UseParallelGC (asserts: true)
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any